If you saw the first FlexASM blog you know we installed and configured FlexASM and a CDB plus a couple of PDBs. Also, this was Policy Managed with a cardinality of 2.  Now let's see what the configuration looks like, and we can break it down using the wonderful crsctl and srvctl tools

First let's ensure we are really running in FlexASM mode:

[oracle@rac02 ~]$ asmcmd showclustermode
ASM cluster : Flex mode enabled


[oracle@rac02 ~]$ srvctl status   serverpool -serverpool naboo
Server pool name: naboo
Active servers count: 2



[oracle@rac01 trace]$ crsctl get node role status -all
Node 'rac01' active role is 'hub'
Node 'rac03' active role is 'hub'
Node 'rac02' active role is 'hub'
Node 'rac04' active role is 'hub'



[oracle@rac01 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       rac01                    STABLE
               ONLINE  ONLINE       rac02                    STABLE
               ONLINE  ONLINE       rac03                    STABLE
               ONLINE  ONLINE       rac04                    STABLE  


You notice that we have 4 ASM listeners one on each node in the Cluster.  You'll see the process as the following on each node:


[oracle@rac01 ~]$ ps -ef |grep -i asmnet

ooracle    6646     1  0 12:19 ?        00:00:00 /u01/app/12.1.0/grid/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit



ora.CRSDATA.DATAVOL1.advm
               ONLINE  ONLINE       rac01                    Volume device /dev/a
                                                             sm/datavol1-194 is o
                                                             nline,STABLE
               ONLINE  ONLINE       rac02                    Volume device /dev/a
                                                             sm/datavol1-194 is o
                                                             nline,STABLE
               ONLINE  OFFLINE      rac03                    Unable to connect to
                                                             ASM,STABLE
               ONLINE  ONLINE       rac04                    Volume device /dev/a
                                                             sm/datavol1-194 is o
                                                             nline,STABLE
The datavol1 ADVM resource runs on all the nodes where indicated it should run.  In this case we are seeing that RAC03 is having some issues.
Let's look into that a little later.  But I like the fact crsctl tells something is amiss here on node3
 
ora.CRSDATA.dg
               ONLINE  ONLINE       rac01                    STABLE
               ONLINE  ONLINE       rac02                    STABLE
               ONLINE  ONLINE       rac03                    STABLE
               OFFLINE OFFLINE      rac04                    STABLE


ora.FRA.dg
               ONLINE  ONLINE       rac01                    STABLE
               ONLINE  ONLINE       rac02                    STABLE
               ONLINE  ONLINE       rac03                    STABLE
               OFFLINE OFFLINE      rac04                    STABLE


The crsdata and fra disk groups resource is started on all nodes except node 4



ora.LISTENER.lsnr
               ONLINE  ONLINE       rac01                    STABLE
               ONLINE  ONLINE       rac02                    STABLE
               ONLINE  ONLINE       rac03                    STABLE
               ONLINE  ONLINE       rac04                    STABLE


We all know, as in 11gR2, that this is the Node listener.


ora.PDBDATA.dg
               ONLINE  ONLINE       rac01                    STABLE
               ONLINE  ONLINE       rac02                    STABLE
               ONLINE  ONLINE       rac03                    STABLE
               OFFLINE OFFLINE      rac04                    STABLE


The pdbdata disk groups resource is started on all nodes except node 4



ora.crsdata.datavol1.acfs
               ONLINE  ONLINE       rac01                    mounted on /u02/app/
                                                             oracle/acfsmounts,ST
                                                             ABLE
               ONLINE  ONLINE       rac02                    mounted on /u02/app/
                                                             oracle/acfsmounts,ST
                                                             ABLE
               ONLINE  OFFLINE      rac03                    (2) volume /u02/app/
                                                             oracle/acfsmounts of
                                                             fline,STABLE
               ONLINE  ONLINE       rac04                    mounted on /u02/app/
                                                             oracle/acfsmounts,ST
                                                             ABLE


ACFS filesystem resource for datavol1 is started on all nodes except node3.
But I think the following has something to do w/ it :-).  Need to debug this a bit later.  I even tried:
[oracle@rac03 ~]$ asmcmd volenable --all
ASMCMD-9470: ASM proxy instance unavailable
ASMCMD-9471: cannot enable or disable volumes



ora.net1.network
               ONLINE  ONLINE       rac01                    STABLE
               ONLINE  ONLINE       rac02                    STABLE
               ONLINE  ONLINE       rac03                    STABLE
               ONLINE  ONLINE       rac04                    STABLE
ora.ons
               ONLINE  ONLINE       rac01                    STABLE
               ONLINE  ONLINE       rac02                    STABLE
               ONLINE  ONLINE       rac03                    STABLE
               ONLINE  ONLINE       rac04                    STABLE


The Network (in my case I only have only Net1) and ONS are same as in previous versions


ora.proxy_advm
               ONLINE  ONLINE       rac01                    STABLE
               ONLINE  ONLINE       rac02                    STABLE
               ONLINE  OFFLINE      rac03                    STABLE
               ONLINE  ONLINE       rac04                    STABLE


Yep, since proxy_advm is not started on node3, the filesystems won't come online….but again, i'll look at that later
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac02                    STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rac03                    STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rac04                    STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       rac01                    169.254.90.36 172.16
                                                             .11.10,STABLE
ora.asm
      1        ONLINE  ONLINE       rac03                    STABLE
      2        ONLINE  ONLINE       rac01                    STABLE
      3        ONLINE  ONLINE       rac02                    STABLE


Since we have the cardinality of 3 ASM instance we have 3 ASM resources active


ora.cvu
      1        ONLINE  ONLINE       rac01                    STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       rac01                    Open,STABLE
ora.oc4j
      1        ONLINE  ONLINE       rac01                    STABLE
ora.rac01.vip
      1        ONLINE  ONLINE       rac01                    STABLE
ora.rac02.vip
      1        ONLINE  ONLINE       rac02                    STABLE
ora.rac03.vip
      1        ONLINE  ONLINE       rac03                    STABLE
ora.rac04.vip
      1        ONLINE  ONLINE       rac04                    STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       rac02                    STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       rac03                    STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       rac04                    STABLE
ora.tatooine.db
      1        ONLINE  ONLINE       rac01                    Open,STABLE
      2        ONLINE  ONLINE       rac02                    Open,STABLE


As we stated above, I specified a Policy Managed database with cardinality of 2, so I have 2 database instances running
--------------------------------------------------------------------------------

Here's some other important supporting info on FlexASm:


[oracle@rac02 ~]$ srvctl config asm -detail
ASM home: /u01/app/12.1.0/grid
Password file: +CRSDATA/orapwASM
ASM listener: LISTENER
ASM is enabled.
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM


[oracle@rac02 ~]$ srvctl status  filesystem
ACFS file system /u02/app/oracle/acfsmounts is mounted on nodes rac01,rac02,rac04


ANd here's what the Database has to say about FlexASM

NOTE: ASMB registering with ASM instance as client 0x10001 (reg:1377584805)
NOTE: ASMB connected to ASM instance +ASM1 (Flex mode; client id 0x10001)
NOTE: ASMB rebuilding ASM server state
NOTE: ASMB rebuilt 2 (of 2) groups
SUCCESS: ASMB reconnected & completed ASM server state


So for the interesting part:
If you notice that ASM is not running node 4:
[oracle@rac02 ~]$ srvctl status  asm -v

ASM is running on rac01,rac02,rac03
[oracle@rac02 ~]$ srvctl status  asm -detail
ASM is running on rac01,rac02,rac03



So, how does a client (ocrdump, rman, asmcmd, etc..) connect to if ASM if there is no ASM on that node.  Well let's test this using asmcmd on node4.  You notice that a pipe is created, a connect string is generated and passed to ASMCMD to connect remotely to ASM2 on node2!!!!


22-Sep-13 12:54 ASMCMD Foreground (PID = 14106):  Pipe /tmp/pipe_14106 has been found.
22-Sep-13 12:54 ASMCMD Background (PID = 14117):  Successfully opened the pipe /tmp/pipe_14106
22-Sep-13 12:54 ASMCMD Foreground (PID = 14106):  Successfully opened the pipe /tmp/pipe_14106 in read mode
NOTE: Executing kfod /u01/app/12.1.0/grid/bin/kfod op=getclstype..
22-Sep-13 12:54 Printing the connection string
contype = 
driver = 
instanceName = <>
usr = 
ServiceName = <+ASM>
23-Sep-13 16:23 Successfully connected to ASM instance +ASM2
23-Sep-13 16:23 NOTE: Querying ASM instance to get list of disks
22-Sep-13 12:54 Registered Daemon process.
22-Sep-13 12:54 ASMCMD Foreground (PID = 14106):  Closed pipe /tmp/pipe_14106.



Due to so many people asking me other methods besides SQLplus for provisioning PDBs; such as OEM, DBCA, etc. In this blog entry I’ll DBCA, just because its simple to show. As I mentioned in my last PDB blog,
the installer DBCA (initial DBA invocation) looks different than the subsequent (post initial db creation).

The main DBCA screen shows the following pages. We will choose Manage Pluggable Database

PDB12c 2013 08 20 17 46 20

Choose the CDB, Note you could have many CDBs on the same Node or RAC cluster

PDB12c 2013 08 30 17 51 59

We choose our PDB that we created in Part 1 of the blog

PDB12c 2013 08 30 17 52 39

Ahh..we gotta open the PDB first. As before:

CDB$ROOT@YODA> alter session set container=pdbobi;
Session altered.

CDB$ROOT@YODA> alter pluggable database pdbobi open;

Pluggable database altered.

or CDB$ROOT@YODA> alter pluggable database all open;

PDB12c 2013 08 30 17 54 22

Now we can Add support for and configure Database Vault. Additionally, Label Security can be configured.
It would have been nice to enable and modify Resource Manager as well other PDB tasks.
But I get the fact that this DBCA is really driven for the PDB operations (plug,unplug, create and destroy PDB).
Bulk of the PDB admin tasks are provided in EM

PDB12c 2013 08 30 18 14 54

Let’s do a new PDB creation for grins 🙂

PDB12c 2013 08 30 18 21 01

Specify the PDB name, storage location, and a default tablespace. Again, it would have been nice to specify a TEMP tablespace too, but that was left out

PDB12c 2013 08 30 18 22 26

Progress ….

PDB12c 2013 08 30 18 23 18

And Completion….Pretty Straightforward

PDB12c 2013 08 30 18 22 53


Once we have installed 12.1 Database Software, we can create the Container Database and the Pluggable Databases. In my case I did a software only install then manually executed DBCA

In this blog entry I’ll show the screens that walk-thru the configuration of the “first” database. I noticed that once DBCA is used to create the initial database, the capability and options (screens) for DBCA are different; i.e., it much more aligned to create/manage additional databases. I’ll show those screens in Part 3 of PDB

So let’s get started by executing
$ $ORACLE_HOME/bin/dbca

Rac01 2013 09 15 22 39 12

Choose Advanced mode for Policy Managed Database or use “Default Configuration”. Being a big promoter of Policy Managed Databases and since I have 4 RAC nodes (my best practice threshold to choose Policy Managed), I’ll choose, that.

Rac01 2013 09 15 22 39 44

I’ll pick a Global Database name and choose PDB option, and also option to choose how many PDBs to create (with prefix)

Rac01 2013 09 15 22 40 30

Pick a Server Pool name, I chose a cardinality of 2

Rac01 2013 09 15 22 40 58

Define the Management Options

Rac01 2013 09 15 22 41 20

Choose the Storage locations

Rac01 2013 09 15 22 43 31

Define Database Vault Owner and also the Separate Account Manager. Note the user name definitions

Rac01 2013 09 15 22 45 31

And now the finish

Rac01 2013 09 15 22 56 41


Oracle Flashback Guaranteed Restore Point Misunderstanding

Nabil Nawaz, Viscosity NA

 

Recently I had a conversation with a DBA that I work with regarding Guaranteed Restore Points (GRP) in Oracle 11gR2. They had the understanding that if a GRP was created regardless if the database flashback feature was on or off then it would only be created with only the flashback logs necessary to flashback or rewind the database back to the GRP point in time. In other words the flashback logs would never grow in size for a GRP since they did not believe those would be needed by Oracle. This conversation came up when they had to wait several minutes to drop a GRP in the database, they did not expect it to take so long since there should only be a limited amount of flashback logs created for the GRP that would need to be dropped.In reality their understanding was not correct at all and here is the reason why, please see below the supporting information for this.

For example if you create a new GRP in your database then the view v$restore_point will be populated as follows, please note the STORAGE_SIZE column which I rounded up to Megabytes(MB) and in this case a new GRP is only 6MB.

GUARANTEE_FLASHBACK_DATABASE STORAGE_SIZE(MB) TIME NAME
YES

6

8/16/2013 11:24

BEFORE_RELEASE

 

Let’s take a look at another GRP that has existed for some time in another database, please note I rounded the STORAGE_SIZE column to Gigabytes(GB) and in this case the GRP is 75GB in size. This shows and proves that when a GRP is created it is small but as activity runs on the database it will grow and in fact ALL flashback logs will be retained since the GRP is created up until it is finally dropped.

 

GUARANTEE_FLASHBACK_DATABASE STORAGE_SIZE(GB) TIME NAME
YES

75

7/2/2013 16:43

BEFORE_REL_1

 

In the database alert log you will see the following output when dropping a GRP with lots of activity since it was created. You will see the message “Deleted Oracle managed file” repeatedly.

Fri Aug 16 11:21:17 2013

Drop guaranteed restore point BEFORE_RELEASE

Deleted Oracle managed file +RECO_EXAD/odssit/flashback/log_1.5780.819737031

Deleted Oracle managed file +RECO_EXAD/odssit/flashback/log_2.42344.819737033

Deleted Oracle managed file +RECO_EXAD/odssit/flashback/log_3.5651.819737037

Deleted Oracle managed file +RECO_EXAD/odssit/flashback/log_4.5322.819737041

Deleted Oracle managed file +RECO_EXAD/odssit/flashback/log_5.51038.819737285

 

Also per the Oracle Support Documentation which also states all flashback logs will be kept to satisfy the restore point.

“If you enable Flashback Database and define one or more guaranteed restore points, then the database performs normal flashback logging. In this case, the recovery area retains the flashback logs required to flash back to any arbitrary time between the present and the earliest currently defined guaranteed restore point. Flashback logs are not deleted in response to space pressure if they are required to satisfy the guarantee.”

Posted in 11g

Steps to enable the bpdufilter on a Cisco 4948 Switch for outside connectivity for Exadata X2

By Nabil Nawaz, Viscosity NA.

We have an Exadata X2 system we are supporting at a managed hosted Datacenter facility that is being supported by me and our company. One fine day in the datacenter the Juniper switch which allows the Exadata system to communicate to the outside world stopped working. Eventually we found out the hosting facility enabled the bpdufilter on the Juniper switch and in turn we needed to do the same setup on out Cisco switch.

Below is a diagram of the highlevel layout of our setup in our datacenter.

Exadata_switch

  • The Exadata X2 Database Machine connects first to the Cisco 4948 Switch.
  • The Cisco switch connects to the Juniper Switch provided by the hosting facility.
  • Juniper Switch is the gateway to outside internet traffic.

  

A BPDU filter what is that?

Bridge Protocol Data Unit’s known also as BPDU’s play a fundamental part in a spanning-tree topology.

The Spanning Tree Protocol (STP) is a network protocol that ensures a loop-free topology for any bridged Ethernet local area network. The basic function of STP is to prevent bridge loops and the broadcast radiation that results from them. Spanning tree also allows a network design to include spare (redundant) links to provide automatic backup paths if an active link fails, without the danger of bridge loops, or the need for manual enabling/disabling of these backup links.

BPDU’s are sent out by a switch to exchange information about bridge ID’s and costs of the root path. Exchanged at a frequency of every 2 seconds by default, BPDU’s allow switches to keep a track of network changes and when to block or forward ports to ensure a loop free topology. A BPDU filter disables spanning-tree which would result in the port to not participate in STP, and loops may occur.

For more information on Spanning Tree Protocol, please refer to the Wikipedia or Cisco documentation links below.

http://en.wikipedia.org/wiki/Spanning_Tree_Protocol

http://www.cisco.com/en/US/docs/switches/lan/catalyst3560/software/release/12.2_55_se/configuration/guide/swstpopt.html#wp1002608

 

Commands to enable bpdu filter.

 

  • ·         Telnet to cisco switch

$ telnet IPADDRESS

  • ·         Enable commandline for switch

telnet> enable

 

  • ·         Prepare to configure switch.

ciscoswitch-ip# configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

ciscoswitch-ip(config)#interface GigabitEthernet1/48

ciscoswitch-ip(config-if)#

  • ·         Enable BPDU filter

ciscoswitch-ip(config-if)# spanning-tree bpdufilter enable

ciscoswitch-ip(config-if)# end

 

  • ·         Save the configuration to the startup configuration.

 

ciscoswitch-ip# copy running-config startup-config

Destination filename [startup-config]?

 

Building configuration…

Compressed configuration from 3889 bytes to 1546 bytes[OK]

ciscoswitch-ip#reload

Proceed with reload? [confirm]

Connection closed by foreign host

 

  • ·         Verify the configuration and BPDU filter is enabled.

ciscoswitch-ip# show running-config

ciscoswitch-ip# show interfaces status

ciscoswitch-ip# show spanning-tree interface GigabitEthernet1/48 portfast

interface GigabitEthernet1/48

media-type rj45

spanning-tree bpdufilter enable


Steps to change the password on a Cisco Switch

By Nabil Nawaz, Viscosity NA

These steps were used to change the password on a Cisco Switch on Exadata X2.

Telnet to cisco switch(IP Address of Switch) $ telnet <IPADDRESS>

·         Enable command line for switch

telnet> enable

·         Prepare to configure switch

ciscoswitch-ip# configure terminal

exapsw-ip(config)#line vty 0 15

exapsw-ip(config-line)#login

·         Change the password

exapsw-ip(config-line)#password newpassword

exapsw-ip(config-line)#login

exapsw-ip(config-line)#end

 

  • ·         Save the changes to the switch

exapsw-ip#write memory

Building configuration…

Compressed configuration from 4001 bytes to 1608 bytes[OK]

exapsw-ip#

 

  • ·         Try logging again to verify password change