Dbvisit – Standby Database Technology Blog

We had an interesting comment from a potential customer recently. Their impression was that Dbvisit was created specifically for Oracle 8i and 9i, at the stage when Data Guard was not yet fully “matured”.

This is not the reason that we created Dbvisit.

Dbvisit was created out of a desire to provide organizations with an easy to use, robust and secure Disaster Recovery solution. In particular we had those customers in mind who had Oracle Standard (SE) and Standard One (SE1) edition databases, and did not want to  upgrade to Enterprise Edition to make use of Data Guard. This has proven to be a successful strategy, as we are now trusted by companies in many parts of the world showing that Dbvisit is a world-class (and affordable) Oracle disaster recovery solution.

The advantages in using Dbvisit are:

1. Cost. You can save on license fees and maintenance fees using Dbvisit in primary and standby server environment, instead of upgrading to Enterprise Edition to utilise Data Guard.
2. Simplicity. Dbvisit is very simple to install and configure. It includes all the tools and utilities to effectively and easily manage the standby environment.
3. Fully featured. Dbvisit matches Data Guard on features such as graceful switchover, RAC, ASM and OMF. Out of the box Dbvisit also includes a number of features which Data Guard does not, like compression, monitoring and encryption for archive log transportation.
4. Automatic Standby Database creation. Dbvisit creates the standby database for you. Simple yet powerful.

For a comparison between Dbvisit and Data Guard please see here: http://www.dbvisit.com/dataguard.php.

Dbvisit works with all versions of Oracle. Starting from Oracle 8i onwards to the latest release of Oracle. In some cases Dbvisit will also work with Oracle 7.3. Dbvisit works with all editions of Oracle from Oracle XE, Standard One, Standard Edition and Enterprise Edition.

Some of our customers have a mix of Enterprise Edition and Standard Edition databases. Rather than use two different DR solutions, they have decided to standardize on Dbvisit for all their databases, so this includes using Dbvisit on Standard Edition and Enterprise Edition. This strategy has been working very well for them.

For customers that are still running Oracle 8i and 9i databases (and there are still quite a few), Dbvisit can be used to protect those databases and future proof their investment as Dbvisit will work with all versions and features of Oracle when they decide to upgrade. Dbvisit works on Windows, Linux and Unix.

We have just released Dbvisit 5.2.24 for all platforms.

This version fixes an issue introduced in 5.2.22 when creating a standby database using a primary database that has a pfile only. Dbvisit does not pick up the correct pfile to transfer to the remote server. This has been fixed. Database using spfiles are not affected.

The full details of this release are:

New features:

1. If SUCCESS_MAILTO is set, then the Dbvisit daily heartbeat emails also go to this email address instead of ADMINS.

Fixes:

1. Dbvisit does not pick up the correct pfile to transfer to the remote server while creating standby database when the primary database does not use an spfile.

The latest version of Dbvisit can be downloaded from http://www.dbvisit.com/download.php

We have just released Dbvisit 5.2.22 for all platforms.

This version allows for a different Standby Database name (ORACLE_SID_DEST) and different admin directories during the creation of the standby database. It also has a number of fixes, a number of which are related to NLS settings of non English.

The full details of this release are:

New features:

1. Dbvisit now allows the Oracle admin directories on standby to be different from the primary database during creation of the Standby Database.

Fixes:

1. Dbvisit does not pick up the correct sequence for RAC on the standby when NLS is set to Spanish.
2. Fix issue with quotes around multiple email addresses: can’t extract address at ..
3. Improve obtaining asmcmd version for Windows.
4. Dbvisit does not pick up the archive log destination when NLS is set to Spanish during graceful switchover.
5. Create standby database may fail in Oracle 8i due to long tablespace names. This has been fixed.
6. Update check and add more tracing to determine if previous Dbvisit process is still running.
7. Update dbvisit help text.
8. If Dbvisit is unable to create Windows Oracle service during creation of standby database, Dbvisit will suggest to create the service manually.
9. Ignore the following SQLPLUS error message when creating Dbvisit repository using dbvisit_setup: Error accessing PRODUCT_USER_PROFILE
10. Dbvisit will detect if ORACLE_HOME_DR is set to a non-existing or not  valid directory on the standby server when creating a standby database and display a warning.
11. Dbvisit may not correctly detect the creation of Oracle Windows service in other languages during creation of standby database. This has been fixed.

The latest version of Dbvisit can be downloaded from http://www.dbvisit.com/download.php

We have released a new online service desk to further improve our support for Dbvisit. Tickets can be created online for any Dbvisit support issue. Tickets can be viewed and tracked at any stage to ensure progress is being made.

Any updates to the tickets will be automatically emailed to the person who created the ticked.

To create a new ticket in our service desk please go to: http://www.dbvisit.com/support.php

We often get asked what is the difference between switchover and failover? The easiest way to understand is that:

Switchover – This is done when both primary and standby databases are available. It is pre-planned.

Failover – This is done when the primary database is NO longer available (ie in a Disaster). It is not pre-planned.

A switchover (or graceful switchover) is a planned role reversal between the primary and the standby databases. This is used when there is a planned outage on the primary database or primary server and you do not want to have extended downtime on the primary database. The switchover allows you to  switch the roles of the databases so that the standby databases now becomes a primary databases and all your users and applications can continue operations on the “new” primary database (on the standby server). During the switchover operation there is a small outage. How long the outage lasts, depends on a number of factors including the network, the number and sizes of the redo logs. The switchover operation happens on both the primary and standby database.

A failover operation is what happens when the primary database is no longer available. The failover operation only happens on the standby database. The failover operation activates the standby database and turns this into a primary database. This process cannot be reversed so the decision to failover should be carefully made. The failover process is initiated during a real disaster or severe outage.

Automatic Failover

Automatic failover is where the software determines when the standby database should be activated to become the new primary database. There are numerous conditions that can occur (ie: network glitches/outages) in any system which theoretically could disrupt communications between the primary and standby sites. Because of the importance of this decision and the number of variances, we believe it is best not to automate this process but to leave it in the hands of a human.

More information on what happens during a failover: http://blog.dbvisit.com/activating-standby-database-failover/

Dbvisit and failover/switchover

In Dbvisit both failover and switchover is possible. The Dbvisit commands are as follows:

Failover:

dbv_oraStartStop activate orcl1

(where orcl1 is the name of the database)

This command is run on the standby server. Now the standby database has been activated and is now the new primary database.

Switchover:

dbv_oraStartStop switchover orcl1

(where orcl1 is the name of the database)

This command is run on both the primary and standby servers. The primary database will become the standby database, and the standby database becomes the primary database. Dbvisit will continue to operate to keep the new standby database up to date from the new primary database. No configuration changes need to be made.

We have just released Dbvisit 5.2.20 for all platforms.

This version allows for passwords to be stored in encrypted form in the Dbvisit Database Configuration (DDC) file.

The full details of this release are:

New features:

1. Add ENCRYPT_PASSWDS variable to specify if all passwords should be stored encrypted in the DDC file or not. To change the setting run dbvisit_setup and choose option 6.
2. Add SWITCHOVER_WAIT_IN_SEC to control how long Dbvisit waits between each checkpoint iteration for graceful switchover. Total time Dbvisit waits for each checkpoint is set by SWITCHOVER_TIMEOUT_IN_SEC.

Fixes:

1. When standby redo logs are using OMF, graceful switchover may fail. This has been fixed.
2. When log_archive_format on Windows contains %S, the sequence number can be truncated by Oracle. Ensure that Dbvisit also truncates the number the same way.
3. Running dbvisit_setup: Update Dbvisit Database configuration (DDC) file may error for older DDC files.
4. Improve error message for ORA-27048.
5. Improve error message for creation of standby when datafile contains spaces.
6. Improve message for error 5005 when standby controlfile does not contain the archive log sequence.

Dbvisit can be downloaded from our website http://www.dbvisit.com.

From the team at Dbvisit we hope you have a fantastic festive season, and a relaxing and enjoyable New Year.

We had our Dbvisit Christmas party and although not everyone was there because our team spans several countries, we had a great time.

2010. The Year Ahead.

We are excited about what’s in store for 2010. Work is already underway on a number of exciting plans and projects, which will bring significant benefit to new and existing customers. So what do we have in mind?

• A web based interface for Dbvisit
• 24×7 support services
• Helpdesk ticketing system

We look forward to communicating with you in 2010.

Warm regards.
the Dbvisit team

Recently, when running Dbvisit with Oracle 11g, we encountered the following error under certain circumstances with one of our standby databases:

 ORA-27048: skgfifi: file header information is invalid

This error suggests that Oracle is possibly trying to use a non-database file as a database file. However we were pretty sure that this was not the case.

As part of normal Dbvisit processing, Dbvisit can issue the statement “SQL>recover standby database …” to bring the standby database up to date. It was at this point that the error was occurring. Our thanks goes to one our team members, Vit Spinka who determined the cause of the error:

When the “recover standby database …” SQL statement is issued in 11g, Oracle scans the entire archive log:
/oracle/app/oracle/flash_recovery_area/<dbname>/archivelog
directory when Flash Recover Area (FRA) is used.

Any files that are not Oracle related will cause the ORA-27048 error. So any archive files that are compressed will cause this error, even if it is an old archive log that is completely irrelevant.
So this error is occurring because there are archives in FRA that are compressed. Uncompressing those archives will stop the error.

With Dbvisit this error only occurs once Graceful Switchover has been initiated twice. It is only at this point that there will be compressed archive logs in FRA on the standby server. Before switchover has occurred, the standby database will not have any archive files in its FRA as the standby database does not produce archive log files. After switchover the new primary (old standby) will start producing archive log files.  Then when another switchover occurs to bring the configuration back to its orginal state, the original standby will have archive log files in its FRA. If those archives are compressed then the ORA-27048 error will occur.

So, what is the solution?
There are several options to solve this issue:

1. Stop Oracle from scanning the FRA by setting the following command in the Dbvisit Database configuration (DDC) file:

    _SQL_ALTER_SESSION_1 = set logsource "/oracle/oraarch/xxxx"

   (It does not really matter what logsource is set to)
   You can add this to the end of the DDC file. Setting this does not have any impact, nor do you lose any Dbvisit functionality. Remember to alway edit the DDC file on the primary server only. Dbvisit will replicate the DDC file to the standby server when changes are detected.

2. Leave the archives uncompressed on the servers. The archives will be still be compressed before transfer.
   Update the DDC file and set the following:

    LEAVE_COMPRESS_SOURCE = No

    LEAVE_COMPRESS_DEST = No

It is always interesting to see how Oracle can modify its behaviour between different releases, and sobering to consider how changes like this really have the potential to catch you out. One of the great benefits of having a team like Dbvisit Support on your side is that they keep up to date with the latest Oracle changes, and are committed to ensuring that Dbvisit functions seamlessly with the database – maintaining the same great high level of protection and continuity for your systems. And this safety net equals peace of mind, for you, the customer.

We have just released Dbvisit 5.2.18 for all platforms.

This release incorporates a number of suggestions from our customers, which will improve usability of the software. We would like to thank those of you who have made suggestions, and we greatly appreciate your feedback.

From this release forward, all Dbvisit production releases will be an even release number (18, 20, 22, etc) and the development or stage releases will be an uneven number (17, 19, 21 etc).
So 5.2.18 is the production release of 5.2.17.

Some of the highlights in this release are:

• Allow for multiple archive log destinations to be managed by Dbvisit.
• Archive logs can now be managed by specifying partial days. For example keep the archive log files for 3.5 days.
• Improve remote checksum processing. If remote checksum process fails, then only checksum process is repeated, not the whole transfer process.
• Allow for Database which has been recovered with point-in-time recovery with resetlogs where SCN may be lower than current SCN.

The full details of this release are:
New features:

1. Allow for multiple archive log destinations in AMM (Archive Management Module) processing (AMM_PROCESS_MULTIPLE_ARCH_DEST).
2. Improve restartability of standby database creation. Standby Database is now shutdown after all the standby database parameters are obtained.
3. Graceful switchover synchronisation process now includes retries if there are network issues.
4. Improve remote checksum processing. If remote checksum process fails, then only checksum process is repeated, not the whole transfer process.
5. Allow for decimal numbers for DAYS_TO_KEEP_ARCHSOURCE and DAYS_TO_KEEP_ARCHDEST. For example 3.5 days.
6. Detect Oracle XE admin location for Linux.
7. Check for Standby archive log destination during graceful switchover. If location does not exist, then exit as switchover will not be successful.
8. Reduce number of trace file that Dbvisit keeps on server:
   NUM_TRACE_TO_KEEP = 100, DAYS_TO_KEEP_TRACE = 10

9. Check that Database is in archivelog mode.
10. Allow for Database which has been recovered with point-in-time recovery with resetlogs where SCN may be lower than current SCN.
11. Allow for “-” in the DDC file. Example: dbv_prod-vm2.env

Fixes:

1. Improve duplicate archive log handling for OMF files. Include a unique identifier to ensure the same file does not get renamed more than once.
2. Add check for ORA-00342: archived log does not have expected resetlogs SCN.
3. If same archive log is needed more than once by standby database for non RAC, ensure archive log can be compressed.
4. Allow for multiple NLS_LANGUAGE setting during RAC processing.
5. Add 24hour time clock to trace file.
6. Fixed calling AMM from dbv_functions -A
7. Improve error message in Windows if during standby database creation the standby database fails to start.
8. Fixed issue if first_change# is displayed as float, then graceful switchover was not successful.
9. Allow for different instance_name to database_name for graceful switchover.
10. Notify if create standby database process is not able to remove file on standby database.
11. Allow for %d log_archive_format.
12. Check and report if “ORA-00332: archived log is too small” is found.

Dbvisit can be downloaded from our website http://www.dbvisit.com.

During recent testing, a datafile from one of our development databases was deleted. At the same time the same datafile was also deleted from the standby database. So this meant that our development primary and standby databases were no longer available.

However Dbvisit came to our rescue, and the functionality we have been recommending to our customers for a number of years now actually saved us! We employed Dbvisit to successfully recreate our primary and standby databases.

We have the following development servers:
Dev servers 1
dbvisit11 – primary database server
dbvisit12 – standby database server for dbvisit11

Dev servers 2
avisit31 – primary database server
avisit32 – standby database server for avisit31

Dev servers 2 are a clone of dev servers 1 with the same databases.
Due to the deletetion of the datafiles, the databases on dbvisit11 and dbvisit12 were gone.

Here is what we did to get the databases back on dbvisit11 and dbvisit12:

1. Logged onto avisit32 and used dbvisit_setup to automatically create a new standby database on dbvisit12.
2. Activated the standby database on dbvisit12 to become a new primary database.
3. Used dbvisit_setup on dbvisit12 to create a new standby database on dbvisit11.
4. Used graceful switchover to switch the roles between dbvisit12 and dbvisit11.

And so in 4 easy steps we had our databases back on our Dev servers 1: dbvisit11 (primary database) and dbvisit12 (standby database).

It is very rewarding to be able to use our own technology to recover a primary and standby database.