Smart Database Technology

As mentioned in a previous blog (ESXi), we have purchased additional hardware for our development environment. We have been using VMware Server for our existing hardware but wanted to go with a bare-metal hypervisor as this is more efficient for running VM’s (Virtual Machine) than running VMware Server.

The 2 options we have been looking at are:

• VMware ESXi 4.0
• Citrix XenServer 5.6

One of the important criteria for choosing, was an easy conversion path from our current VM’s (using VMware Server) to the new solution.

VMware ESXi 4.0.
The natural choice was to go with VMware ESXi 4.0 as an upgrade to VMware Server. The installation and configuration was easy and we successfully converted our existing VM’s to ESXi using the Converter Standalone Client . We installed ESXi on a USB stick which can then be used to boot the server to run ESXi. This gives the ability to clone the USB stick for DR and backup purposes.

The management console using vSphere Client is also easy to use, although some features are hidden away and take some time to find (ie. uploading and downloading files to the datastore). With vSphere Client you get a nice overview of the VM’s and the resources used by each VM.

We were all set to go with deploying VMware ESXi on our hardware and the last step was checking the security and tightening the security of ESXi. We deploy some of our servers on public facing networks and as such we need to be very careful about security and we have some strict policies around security.

This is where ESXi was not secure enough for us. It is not possible to change any of the default ESXi ports used by the management console and it is also not possible to run a firewall service on ESXi. The operating system that runs on the host, is a cut down Linux version with limited possibilities for any configuration changes and additions. This also makes it difficult to use standard methods for VM backups such as rsync and scp.

VMware states that the ESXi management console interface should not be on a public network, and/or should have a firewall in front of it. Having a firewall in front is still not secure enough for us, we want the ability to change the default ports that the management console communicates on, otherwise those ports will still get hammered even if they are behind a firewall.  With our configuration it would be possible to have the management console on a private internal network and have one of the VM’s connect to this private network. This is a bit of a chicken and egg situation where the VM has to be running in order to manage the VM’s. Not an ideal situation for us.

Citrix XenServer 5.6
The other option was to try Citrix XenServer. We had a few issues installing XenServer on a USB stick. Turns out a 16GB USB stick needs to be used, smaller sizes are possible but require tweaking on the install scripts.

XenServer is a nice solution and the XenServer management Console called XenCenter is great. With XenServer you get a full blown Linux operating system allowing you to to secure the host and do regular VM backups.

XenServer works quite differently to ESXi, and although not being experts in this area, we understand it as follows: XenServer is more “bare bones” than ESXi. The VM’s on XenServer talk more directly to the hardware whereas with ESXi there is more of an abstraction layer. The advantage for XenServer is that it runs the VM’s faster and more efficient than ESXi. The downside is that each OS has to be certified to run as a VM in XenServer as it has to talk directly to the hardware and specific drivers have to be build. This means each new release of Linux has to be certified against XenServer before being able to create a VM for that specific OS. With ESXi (and other VMware solutions), the VM uses generic VMware drivers so the OS is not so crucial. For example, we are running Solaris x86 and older versions of Linux in VMware Server.

To convert our existing VMware Server VM’s to XenServer, we use Citrix XenConvert. We first imported the VM’s to ESXi and then exported the VM’s out of ESXi using the Open Virtualization Format (OVF) and then converted it to XenServer. That was where we started to have issues with XenServer.

If the VM had only one virtual disk, then the conversion was fine, but as soon as the VM had more than one virtual disk the conversion was not successful. With a lot of tweaking we may have been able to improve the conversion but our VM’s are quite complicated systems and we did not want to find out three months down the track that one of the components in our VM was not working due to a conversion error.
The MS Windows server VM that we had converted to XenServer also required new activation because Windows detected a significant change in the underlying hardware.

Also the limited ability to only run specific OS as VM’s was also a limiting factor for XenServer. Our software Dbvisit runs on many different platforms and we need to be able to test on all of them.

Conclusion
What did we decide in the end to use? Who is the winner between VMware ESXi versus XenServer?

We have gone back to using VMware Server. While there is a performance penalty, you can install the OS of your choice and harden it as much as you like.
ESXi 4.0 is not secure enough for our configuration and it was too difficult to convert our existing VM’s to XenServer.

Had we started out from scratch and did not have any existing VM’s, then the clear winner would be XenServer for us.

Great news! Dbvisit has been elevated to the level of Gold Partner (Specialized) status by Oracle in recognition of our hard work in crafting a product that specializes in the area of Oracle Disaster Recovery.

To achieve this Dbvisit had to demonstrate “competency development, business results, expertise and proven success to be recognized by Oracle, and preferred by customers.*” For Dbvisit these values underpin everything we do and are vital to delivering a product that Oracle users want and prefer.

Thanks Oracle, the recognition is well received.

(* source: http://blogs.oracle.com/certification/2009/12/0127.html)

We have purchased additional hardware for our test and development environment. We have been using VMware server for our other dev/test hardware but wanted to try VMware ESXi.

The software you will need is:

• VMware ESXi 4.0 (to run on the new hardware)
• VMware vSphere Client with vSphere Host Update Utility (to run on the client which will manage the ESXi)
• VMware vSphere Command-Line Interface – Windows Installer (to run on the client which will manage the ESXi)
  Command-line tools to manage your vSphere infrastructure.
• VMware vCenter Converter Standalone (to convert VMware server guests to ESXi)

We wanted to install VMware ESXi on a USB stick. This gives the advantage of easily creating copies of the USB stick (using Linux dd command) in case ESXi did not want to boot. It also allows us to create a clone of the whole ESXi server.

Moving from VMware server to ESXi requires quite a different approach as you no longer are running a full OS on the host. All the things you take for granted by having a full OS are no longer there and this takes a bit of getting used to. You are also no longer as flexible because simple OS command backups do not work or are limited in their features.

The setup of ESXi is very straightforward and easy. However we quickly found one of the short comings to ESXi. Our USB had decided not to work anymore and as we were still in setup phase we had not created a backup yet.

To fix it, we rebooted the server with the ESXi CD and this allowed us the repair the USB stick and bring back ESXi. However all the meta data had gone. The VMware guests where still on the disk, but the vSphere Client did not pickup any of the existing VMware guests or resource pools that we had created. There is no quick refresh button.

This would be a major blow if this was a production server.

However we did find a way to re-register an existing VMware guest using the vSphere CLI tool. This tool is a collection of Perl scripts that are installed on your local machine (can be Windows or Linux) and connects to ESXi to run certain commands.

To re-register a VMware guest, run the following command:

perl vmware-cmd -U <myuser> -P <mypassword> -H HostABC
    -s register /vmfs/volumes/datastore1/MyVM/MyVM.vmx

It helps to know the path of the vmx. This can be found out by logging onto the console of the ESXi server. Typing ALT-F1 and then typing “unsupported”. This brings you to the console of the ESXi server and you can use use find and ls commands to find the location of the vmx file.

This brings back the VMware guests but does not bring back the resource pools. However in the vSphere Client you can re-create the same resource pools and move the VMware guests under the resource pools.

Before we deploy our server we want to make a backup of its configuration so that if something similar happened we can restore the configuration. To do this run the following command from CLI client:

C:\Program Files\VMware\VMware vSphere CLI\bin>perl vicfg-cfgbackup.pl --username root
--password xxxx --server xxxx -s C:\backup\server_esxi_config.dat
Saving firmware configuration to c:\backup\server_esxi_config.dat ...

Other handy commands to run under VMware ESXi console:
View disks with:

fdisk -l

View current disk paths with:

esxcfg-mpath -l

Other handy commands to run under vSphere CLI:
To identify the working directory of the virtual machine from ESXi and the vSphere CLI, run the command:

vmware-cmd.pl -H <host> -U <username> -P <password> -l

To create a folder for the copy of the virtual machine from ESXi and the vSphere CLI, run the command:

   vifs.pl -H <host> -U <username> -P <password> --mkdir '[datastore] dir'

where datastore is the name of the datastore, and dir is the name of the new directory

We have been working hard on a web based version of Dbvisit and although it is not yet finished we want to give a sneak preview of what the Dbvisit web based version will look like. Even though there will be a new GUI interface, the command line functionality will still be the same as with the current version if you prefer to use command line style.

The Dbvisit web based version will feature its own build in web server and will not rely on any external software, and is not build in APEX. The Dbvisit web based version will be platform independent and will work on Windows, Linux and Unix.

The Dbvisit web version main menu looks like this:

The menu items are easy to understand and are kept simple.

Click on “Setup” and the Dbvisit setup and configuration is an easy to use wizard which has most of the options already filled in and lets you know at which stage of the process you are:

Although Dbvisit is a background batch process, it can be run interactively through the web based version for testing purposes:

The “Standby Server” tab can be selected to run Dbvisit commands on the standby server.

The Dbvisit log files can also be easily displayed through the web based version. It will even show the Oracle Alert log:

As part of the web based version, Dbvisit will show details of the last transfer:

The sub menu provides additional functionality, which includes creating the standby database:

There will also be a reporting function which will report KPI’s like the log gap reports, time taken for the transfer process, size of the log files etc.

We are very excited about this release. There is still more development and testing to do before it can be released. There is no release date set, but as soon as we have a date, we will publish it here.  This version will be released as Dbvisit version 6.0.

We had an interesting comment from a potential customer recently. Their impression was that Dbvisit was created specifically for Oracle 8i and 9i, at the stage when Data Guard was not yet fully “matured”.

This is not the reason that we created Dbvisit.

Dbvisit was created out of a desire to provide organizations with an easy to use, robust and secure Disaster Recovery solution. In particular we had those customers in mind who had Oracle Standard (SE) and Standard One (SE1) edition databases, and did not want to  upgrade to Enterprise Edition to make use of Data Guard. This has proven to be a successful strategy, as we are now trusted by companies in many parts of the world showing that Dbvisit is a world-class (and affordable) Oracle disaster recovery solution.

The advantages in using Dbvisit are:

1. Cost. You can save on license fees and maintenance fees using Dbvisit in primary and standby server environment, instead of upgrading to Enterprise Edition to utilise Data Guard.
2. Simplicity. Dbvisit is very simple to install and configure. It includes all the tools and utilities to effectively and easily manage the standby environment.
3. Fully featured. Dbvisit matches Data Guard on features such as graceful switchover, RAC, ASM and OMF. Out of the box Dbvisit also includes a number of features which Data Guard does not, like compression, monitoring and encryption for archive log transportation.
4. Automatic Standby Database creation. Dbvisit creates the standby database for you. Simple yet powerful.

For a comparison between Dbvisit and Data Guard please see here: http://www.dbvisit.com/dataguard.php.

Dbvisit works with all versions of Oracle. Starting from Oracle 8i onwards to the latest release of Oracle. In some cases Dbvisit will also work with Oracle 7.3. Dbvisit works with all editions of Oracle from Oracle XE, Standard One, Standard Edition and Enterprise Edition.

Some of our customers have a mix of Enterprise Edition and Standard Edition databases. Rather than use two different DR solutions, they have decided to standardize on Dbvisit for all their databases, so this includes using Dbvisit on Standard Edition and Enterprise Edition. This strategy has been working very well for them.

For customers that are still running Oracle 8i and 9i databases (and there are still quite a few), Dbvisit can be used to protect those databases and future proof their investment as Dbvisit will work with all versions and features of Oracle when they decide to upgrade. Dbvisit works on Windows, Linux and Unix.

We have released a new online service desk to further improve our support for Dbvisit. Tickets can be created online for any Dbvisit support issue. Tickets can be viewed and tracked at any stage to ensure progress is being made.

Any updates to the tickets will be automatically emailed to the person who created the ticked.

To create a new ticket in our service desk please go to: http://www.dbvisit.com/support.php

From the team at Dbvisit we hope you have a fantastic festive season, and a relaxing and enjoyable New Year.

We had our Dbvisit Christmas party and although not everyone was there because our team spans several countries, we had a great time.

2010. The Year Ahead.

We are excited about what’s in store for 2010. Work is already underway on a number of exciting plans and projects, which will bring significant benefit to new and existing customers. So what do we have in mind?

• A web based interface for Dbvisit
• 24×7 support services
• Helpdesk ticketing system

We look forward to communicating with you in 2010.

Warm regards.
the Dbvisit team

During recent testing, a datafile from one of our development databases was deleted. At the same time the same datafile was also deleted from the standby database. So this meant that our development primary and standby databases were no longer available.

However Dbvisit came to our rescue, and the functionality we have been recommending to our customers for a number of years now actually saved us! We employed Dbvisit to successfully recreate our primary and standby databases.

We have the following development servers:
Dev servers 1
dbvisit11 – primary database server
dbvisit12 – standby database server for dbvisit11

Dev servers 2
avisit31 – primary database server
avisit32 – standby database server for avisit31

Dev servers 2 are a clone of dev servers 1 with the same databases.
Due to the deletetion of the datafiles, the databases on dbvisit11 and dbvisit12 were gone.

Here is what we did to get the databases back on dbvisit11 and dbvisit12:

1. Logged onto avisit32 and used dbvisit_setup to automatically create a new standby database on dbvisit12.
2. Activated the standby database on dbvisit12 to become a new primary database.
3. Used dbvisit_setup on dbvisit12 to create a new standby database on dbvisit11.
4. Used graceful switchover to switch the roles between dbvisit12 and dbvisit11.

And so in 4 easy steps we had our databases back on our Dev servers 1: dbvisit11 (primary database) and dbvisit12 (standby database).

It is very rewarding to be able to use our own technology to recover a primary and standby database.

Oracle Open World 2009 is now behind us. It is a massive event that attracted 32,000 people and had about 1,900 sessions. It is very well organised by Oracle and partners. It is held in 3 huge convention centres that are across the road from each other. One road is permanently closed to traffic as they have erected big tents to use as lunch and entertainment venue. Almost the whole city of San Francisco is taken over by OOW as you see delegates walking with badges on the street wherever you are in San Francisco.

It was nice to meet up again with old friends and it was also a privilege to meet so many new Oracle DBA’s and Developers. OOW is certainly a place to bring the worlds best Oracle specialists together in one place. It is a great opportunity for networking and sharing knowledge and ideas.

My two presentations went very well and had a good turnout. One presentation was on Disaster Recovery and another on Perl. For details of my presentations and to download the slides and “homework” please go to OOW2009.

I think the Perl presentation hit a chord with many DBA’s as it is a highly effective tool for DBA’s, one that is too often overlooked, and one that many DBA’s would like to learn. But as with many programming languages you need someway to get you started, to learn the fundamentals and for someone to show you the right way to learn the language so that you do not start off with “bad” habits. This presentation was about how to start learning Perl and to go over some of the language fundamentals and best practices. The presentation was very well received as I received many positive comments from the audience. I was asked to submit this presentation for Collaborate 10 in Las Vegas next April which I now have done.

The Wednesday night party had everyone (all 32,000) transported in buses to an island in San Francisco Bay that is reachable by bridge. This was a huge party with 2 large stages. The entertainment was Aerosmith, Roger Daltry, Three Dog Night and The Wailers. What a great party and great food and drinks! I think Oracle hired all the buses, taxis and limos that are available in San Francisco.

What was the top new feature of Oracle 11gR2. For me this is “Edition-based redefinition”. As Tom Kyte said in his presentation: “It is the killer feature Of Oracle Database 11g Release 2. It is worth 2 features!”. It is like version control within the database. Not version control on the data itself but version control on the schema objects. The schema objects types that are editionable include Synonyms, Views and PL/SQL object types (like triggers, functions, procedures etc). So tables are not included in this.

Edition-Based Redefinition or EBR allows significantly less downtime when you are upgrading applications as you no longer need to wait for all the users to stop using the application before upgrading. Currently if releasing new PL/SQL code, the code needs to be recompiled but you cannot recompile until all users have stopped using the code. There are always a lot of dependencies when compiling PL/SQL, so basically it means no one can use the application when this is happening. With EBR, you can now compile the new code without affecting the current application. EBR is available in all editions of Oracle 11gR2.

In summary it was an event that was well worth attending and it was a privilege to be presenting at such a great event. Hope to see you there next year!

Every time I use Perl, I am amazed at this programming language created by Larry Wall. I have used several languages in the past, but Perl is now my favourite. It takes a little getting used to, but once you are used to it, it is great.

What makes it so great

• CPAN site. Small and large Perl modules that can be easily plugged into your code to make life easier. Some of the modules available are: logging, email, html template engines etc. There are over 70,000 modules.
• Runs on Windows, Linux, Mac and Unix as well as other OS. It is truly a multi-platform language. For each OS you do have to take certain precautions, but there are CPAN modules that allow you to deal with each OS’es peculiarities like file name conventions.
• It can be used in OO style but also conventional style.
• It is very efficient. With very few lines of code, a lot can be achieved.

I have read the following negative things being said about Perl:
- “Good for little programs, but not for large programs because you get spaghetti code.”
I think this is not true, in any language you can write spaghetti code. This is not due to the programming language but due to the programming style.

- “Things can be done in more than one way”
I think this can be a good thing or a bad thing. With Perl, I consider this a good thing as it demonstrates the power and flexibility of the language.
True this can mean that Perl can be a bit cryptic as you may be used to seeing things solved another way. However it is a great language if you are a perfectionist and always keen to learn new things. With Perl there is almost always a better way to do things and you learn all the time. At least I do.

True, Perl is not so trendy anymore as say Python, Ruby or Java. It is also cometimes difficult to compare different languages as some are more suited to a particular application than others. You also have to look at how well the language is known and how many resources are available. For example to build a website I would probably choose PHP over Perl as there are so many more PHP programmers in the world than Perl programmers ( I also like PHP).

If you are an Oracle DBA that likes to automate specific tasks, I would highly recommend using Perl. I used to do everything in Korne Shell, and I thought it was a great language. But once you learn Perl, you realise your hands are no longer tied behind your back!

Perl is great to automate some of the tasks that may be required in your disaster recovery plan. This can be changing tnsnames.ora entries, or DNS entries.

You are not in isolation if you use Perl in your organisation. Oracle 11g uses Perl to write some of their interfaces. For example asmcmd (and asmcmdcore) are witten in Perl. Vmware uses Perl for their installation routine.

CPAN

To install a cpan module, type cpan (in Windows use PPM. Start cmd and type ppm):

CPAN>i /mailer/

The i is for information, this will list all modules with the word “mailer”.

To install a particular module, you have to specify the name exactly as it appears:

CPAN> install Mail::Mailer
That’s it and you can use it.

How to learn Perl

The best way to learn Perl is to start with a small project. For example write a little Perl program that scans the Oracle alert log and send an email if an error appears. You will probably have Perl installed already because Perl comes pre-installed on Unix and Linux. On Windows go to activestate.com and download Perl.

The best Perl book in my opinion is Programming Perl by Larry Wall, Tom Christiansen, Jon Orwant.

So thanks to Larry Wall and all the great people that help support Perl and CPAN.