DPM and Oracle (or any other VSS-capable, not officially supported datasource)

I know it's been asked before and there haven't been any real solutions. Impossible is not an answer, suggesting to buy a 3rd party app is ridiculous, shutting down Oracle during backup is even more so. Are there really no better answers? Oracle supports VSS, why the hell was VSS invented if DPM can't use that?

Can someone think of a solution? My least stupid idea was to create a VSS snapshot and expose it as a volume, then protect it with DPM, but I haven't even tried it yet because I now believe it wouldn't work (DPM can't use the USN journal on a VSS snapshot). Anyway there might be some other tricky solutions like this one.

The question doesn't have to be Oracle specific, let's just say I have any VSS-capable data source and want to protect that, and the data is huge so I would really love to use the express full backup feature of DPM as well. It's okay if DPM reads through all the data, I just don't want it to transport everything through the wire, and that's exactly what e.f.b. is.

Has anyone seen the 3rd party solution by i365? Does it really use the Oracle VSS writer and back up data with express full backup? If so, and DPM is extensible like that, then I'm sure I can do that, too, any pointers?

December 8th, 2011 9:22pm

Generic VSS capable data source backup support is being added as part of  DPM 2012.

But the only supported Data source as of now is  SC VMM 2012.

Thanks,

Satish

[This posting is provided "AS IS" with no warranties, and confers no rights.]

Edited: Made corrections after   the miscommunication earlier.

Free Windows Admin Tool Kit Click here and download it now
December 8th, 2011 10:32pm

Generic VSS capable data source backup support is being added as part of  DPM 2012.

But the only supported Data source as of now is  SC VMM 2012.

Thanks,

Satish

[This posting is provided "AS IS" with no warranties, and confers no rights.]

Edited: Made corrections after   the miscommunication earlier.

December 8th, 2011 10:32pm

Yes, Generic VSS Capable Datasource backup support is being added as part  DPM 2012.

It is the same thing that you are  looking for.


That sounds promising. I read about it and multiple sources write something like this: "Backup solutions for any Microsoft app with a VSS writer ". Are you saying that I will surely be able to select the Oracle VSS writer from some drop-down list and select which folders to protect with that VSS writer? I think that's the only two information needed to work with a generic datasource. I hope there will be no limitations that I can only select Microsoft's VSS writers.

By the way, just out of curiosity: if for some reason it (or some other data source we encounter) doesn't work with DPM, are there public API's to extend DPM, like i365 does, or do they use undocumented (but supported) API's?



  • Edited by fejesjoco Friday, December 09, 2011 1:11 PM
Free Windows Admin Tool Kit Click here and download it now
December 9th, 2011 8:35am

Yes, Generic VSS Capable Datasource backup support is being added as part  DPM 2012.

It is the same thing that you are  looking for.


That sounds promising. I read about it and multiple sources write something like this: "Backup solutions for any Microsoft app with a VSS writer ". Are you saying that I will surely be able to select the Oracle VSS writer from some drop-down list and select which folders to protect with that VSS writer? I think that's the only two information needed to work with a generic datasource. I hope there will be no limitations that I can only select Microsoft's VSS writers.

By the way, just out of curiosity: if for some reason it (or some other data source we encounter) doesn't work with DPM, are there public API's to extend DPM, like i365 does, or do they use undocumented (but supported) API's?



  • Edited by fejesjoco Friday, December 09, 2011 1:11 PM
December 9th, 2011 8:35am

I would like to clarify a bit on this.  As part of DPM 2012 Generic Data Source initiative, DPM 2012 will protect VMM 2012.  No plans to enable backup solutions for the other applications.  Sorry for the confusion.

 

Thanks,

Satish

Free Windows Admin Tool Kit Click here and download it now
December 9th, 2011 11:46am

That's very sad :(

Then can you give me pointers on how to extend DPM with a custom VSS writer?

December 9th, 2011 1:12pm

Hi,

take a look here for Oracle backup and DPM 2010... collected by Mike.

http://social.technet.microsoft.com/Forums/en-US/dpmsqlbackup/thread/d43da93f-5309-4980-a24d-2e74c7460e43

Regards

Martin

Free Windows Admin Tool Kit Click here and download it now
December 15th, 2011 3:42pm

So your problem is to protect Oracle database.

 

The oracle can be protocted in two ways (Online and Offline). It depends on the status of your Oracle DB wheather its NoArchiveLog mode (offline backup)or Archive log mode (online backup)

 

There is a script for backuping Oracle db under Oracle folder (depend on which oracle version you are running)  usually it will be under oraclexe\app\oracle\product\10.2.0\server\BIN\backup.bat

 

This will allow to backup Oracle DB to your local HD using flash recovery area (you have to make sure that you have enough space)

Schedule the DPM full backup after Oracle full backup and copy the files to your DPM server.

 

Hope that helps,

Laith.

____________________________________________

If you found this helpful please mark it as an Answer.

  • Proposed as answer by Laith_IT Wednesday, December 21, 2011 8:10 PM
  • Unproposed as answer by Laith_IT Thursday, December 22, 2011 9:58 AM
  • Proposed as answer by Laith_IT Wednesday, February 01, 2012 12:45 PM
December 16th, 2011 7:18am

So your problem is to protect Oracle database.

 

The oracle can be protocted in two ways (Online and Offline). It depends on the status of your Oracle DB wheather its NoArchiveLog mode (offline backup)or Archive log mode (online backup)

 

There is a script for backuping Oracle db under Oracle folder (depend on which oracle version you are running)  usually it will be under oraclexe\app\oracle\product\10.2.0\server\BIN\backup.bat

 

This will allow to backup Oracle DB to your local HD using flash recovery area (you have to make sure that you have enough space)

Schedule the DPM full backup after Oracle full backup and copy the files to your DPM server.

 

Hope that helps,

Laith.

____________________________________________

If you found this helpful please mark it as an Answer.

  • Proposed as answer by Laith_IT Wednesday, December 21, 2011 8:10 PM
  • Unproposed as answer by Laith_IT Thursday, December 22, 2011 9:58 AM
  • Proposed as answer by Laith_IT Wednesday, February 01, 2012 12:45 PM
Free Windows Admin Tool Kit Click here and download it now
December 16th, 2011 7:18am

 (you have to make sure that you have enough space)
I don't have enough freespace, far from it.
December 23rd, 2011 5:08pm

You can change the flash recovery area in Oracle. Connect a USB, ISCSI disk to the server and problem solved.

 

____________________________________________

If you found this helpful please mark it as an Answer.

Free Windows Admin Tool Kit Click here and download it now
December 27th, 2011 12:14pm

I don't want to back up to an external disk, I want to back up to DPM.

So, with DPM 2010, the best I can do is a pre-backup script that stops Oracle and a post-backup script to start Oracle. This seems to work for now.

And now I'm waiting for DPM 2012 to see if I can customize it to use Oracle's VSS writer...

  • Marked as answer by fejesjoco Wednesday, February 01, 2012 1:25 PM
February 1st, 2012 1:25pm

I don't want to back up to an external disk, I want to back up to DPM.

So, with DPM 2010, the best I can do is a pre-backup script that stops Oracle and a post-backup script to start Oracle. This seems to work for now.

And now I'm waiting for DPM 2012 to see if I can customize it to use Oracle's VSS writer...

  • Marked as answer by fejesjoco Wednesday, February 01, 2012 1:25 PM
Free Windows Admin Tool Kit Click here and download it now
February 1st, 2012 1:25pm

There are many things that Microsoft do not put on the 'support statement' for their products, the technical ability and capability is frequently different from this.

I understand that Microsoft will not provide support on Oracle or any generic data source because they did not make the generic APP, nor does Microsoft have a support agreement or intend to have a support agreement with Oracle or other third parties so they can follow through on support.

That is understood.

What we want is what was quoted as 'creating and XML file' for a custom application with a VSS writer, we want that documentation on the creation, placement and registration of that XML file with some examples.

The XML file is supposed to, and I quote
Allows basic protection/recovery support of any referential data source
Supports Full Application Backup (Express Full, Delta Replication and Consistency Check) for any application
Supports Original location recovery & Restore as Files to a network location
Capability to use Xml to support applications which do not have a VSS writer


I understand that any custom app that we create an XML file is unsupported, but last I checked on the technet http://technet.microsoft.com/library/hh758173.aspx there was no documentation on this feature - at all - and this is considering it was a major feature that was advertised repeatedly in the lead up to release, and all but

To be clear Wicresoft/Microsoft - I'm asking for DPM 2012 feature support/documentation, not 3rd party product support.

After a bit of looking around I finally found in the teched  SIM341 video @ 47:23 http://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/SIM341 the VSS register command for generic data sources which is

Set-DPMGlobalProperty -RegisteredWriters @('VSSWriterID') -DPMservername DPMServerName

I've successfully registered a test Oracle DB in our environment, and can see it and the DPM protection group data source now after using the below doc to register a Test DB SID with the Oracle VSS writer in addition to the PS command above.

http://docs.oracle.com/cd/B28359_01/win.111/b32010/vss.htm

After getting all that done I appear to have done a successful backup of our little test Oracle database using DPM2012 with the RC...I'm currently downloading the RTM, but it shouldn't be any different.

It's fairly painless to get going with 11gr2 fejesjoco<abbr class="affil"></abbr>, see how you go.

In short for Oracle DB in a very very quick test which I'll need to get verified by my Oracle DBA colleague.

  1. Install VSS writer into the desired SID ie. C:\>oravssw TESTDB /i /user:.\oracle /password:********
  2. use C:\>vssadmin list writers
  3. copy the writer ID listed against your SID
  4. Set-DPMGlobalProperty -RegisteredWriters @('VSSWriterID') -DPMservername DPMServerName
  5. Create a protection group, select the Oracle VSS Writer - SID and the object under it
  6. Watch Microsoft backup an Oracle product with a grin on your face

When you do a restore for the source I can see the following items:

  • All tablespaces (with granular DBF restore)
  • Control File
  • Flash Recovery Area
  • Server Parameter File

Wicresoft/Microsoft, please get on to the product team and hastle them for documentation.

I'm going to play with the RTM tomorrow, I should get the same results. I'll put another post otherwise with any notes from my Ora DBA colleague.

Regards,

David

  • Proposed as answer by Bob Scratchit Wednesday, February 06, 2013 11:43 AM
April 12th, 2012 3:43pm

There are many things that Microsoft do not put on the 'support statement' for their products, the technical ability and capability is frequently different from this.

I understand that Microsoft will not provide support on Oracle or any generic data source because they did not make the generic APP, nor does Microsoft have a support agreement or intend to have a support agreement with Oracle or other third parties so they can follow through on support.

That is understood.

What we want is what was quoted as 'creating and XML file' for a custom application with a VSS writer, we want that documentation on the creation, placement and registration of that XML file with some examples.

The XML file is supposed to, and I quote
Allows basic protection/recovery support of any referential data source
Supports Full Application Backup (Express Full, Delta Replication and Consistency Check) for any application
Supports Original location recovery & Restore as Files to a network location
Capability to use Xml to support applications which do not have a VSS writer


I understand that any custom app that we create an XML file is unsupported, but last I checked on the technet http://technet.microsoft.com/library/hh758173.aspx there was no documentation on this feature - at all - and this is considering it was a major feature that was advertised repeatedly in the lead up to release, and all but

To be clear Wicresoft/Microsoft - I'm asking for DPM 2012 feature support/documentation, not 3rd party product support.

After a bit of looking around I finally found in the teched  SIM341 video @ 47:23 http://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/SIM341 the VSS register command for generic data sources which is

Set-DPMGlobalProperty -RegisteredWriters @('VSSWriterID') -DPMservername DPMServerName

I've successfully registered a test Oracle DB in our environment, and can see it and the DPM protection group data source now after using the below doc to register a Test DB SID with the Oracle VSS writer in addition to the PS command above.

http://docs.oracle.com/cd/B28359_01/win.111/b32010/vss.htm

After getting all that done I appear to have done a successful backup of our little test Oracle database using DPM2012 with the RC...I'm currently downloading the RTM, but it shouldn't be any different.

It's fairly painless to get going with 11gr2 fejesjoco<abbr class="affil"></abbr>, see how you go.

In short for Oracle DB in a very very quick test which I'll need to get verified by my Oracle DBA colleague.

  1. Install VSS writer into the desired SID ie. C:\>oravssw TESTDB /i /user:.\oracle /password:********
  2. use C:\>vssadmin list writers
  3. copy the writer ID listed against your SID
  4. Set-DPMGlobalProperty -RegisteredWriters @('VSSWriterID') -DPMservername DPMServerName
  5. Create a protection group, select the Oracle VSS Writer - SID and the object under it
  6. Watch Microsoft backup an Oracle product with a grin on your face

When you do a restore for the source I can see the following items:

  • All tablespaces (with granular DBF restore)
  • Control File
  • Flash Recovery Area
  • Server Parameter File

Wicresoft/Microsoft, please get on to the product team and hastle them for documentation.

I'm going to play with the RTM tomorrow, I should get the same results. I'll put another post otherwise with any notes from my Ora DBA colleague.

Regards,

David

  • Proposed as answer by Bob Scratchit Wednesday, February 06, 2013 11:43 AM
Free Windows Admin Tool Kit Click here and download it now
April 12th, 2012 3:43pm

Amazing news!

Please tell me something. Can you confirm that the Oracle VSS writer is used by DPM? I don't know if Oracle logs the fact or if you can find the writers associated with a VSS snapshot. Also, is it an express full backup? I mean, does it always transfer the whole data or only blocks that changed? It's easy to find out if you create a few huge tablespaces and measure the time it takes for the initial and the second backup.

April 12th, 2012 3:55pm

Well done , Amazing.  I will be testing this tomorrow

Sean

Free Windows Admin Tool Kit Click here and download it now
April 25th, 2012 11:18pm

Hi Fejesjoco,

Yes the Oracle VSS writer is used by DPM, from the Oracle logs as well as the windows logs everything points to the Oracle VSS writer coming into use.

On the second question, yes! It does do an express full, so it is very fast transferring only the changed blocks.

During my testing I found that when the database was under HEAVY load (proportional to the IO capability/RAM/CPU of the VM) the Oracle VSS writer would fail in creating the snapshot as it could not quiesce all the databases within the given time.

I haven't had the time or large amount of hardware to do 'production load/hardware' testing, so I would be interested to see how other people go backing up Oracle in a heavy dev or production environment with DPM2012.

One downside of using the Oracle VSS writer is that it doesn't appear you have control over which tablespaces are backed up, it's an all or nothing method.

With MSSQL server they suggest not backing up more than 10DBs at the time due to quiescing timeing issues, and a hard limit of 50. So equally with the Oracle VSS writer you'd have to look at the number of tablespaces/activity and if you can quiesce in time given your performance capability of the machine.

I did notice when I was smashing the Oracle DB with inserts at the same time as trying to perform a backup, that the small VM's RAM was exhausted as the Oracle VSS writer service was taking up to 750MB of RAM, so that may have been a cause for the failed backup as well. Depending on the behavior other people see on their DPM with Oracle VSS writer they may need to set aside RAM for the Oracle VSS writer.

So people see the steps for setting this up, can you bump this up as an answer?

Install VSS writer into the desired SID ie. C:\>oravssw TESTDB /i /user:.\oracle /password:********

  1. use C:\>vssadmin list writers
  2. copy the writer ID listed against your SID
  3. Set-DPMGlobalProperty -RegisteredWriters @('VSSWriterID') -DPMservername DPMServerName
  4. Create a protection group, select the Oracle VSS Writer - SID and the object under it
  5. Watch Microsoft backup an Oracle product with a grin on your face

When you do a restore for the source I can see the following items:

  • All tablespaces (with granular DBF restore)
  • Control File
  • Flash Recovery Area
  • Server Parameter File

Kind Regards,

David

  • Proposed as answer by DBR-9 Tuesday, May 15, 2012 9:42 AM
  • Marked as answer by fejesjoco Tuesday, May 15, 2012 9:55 AM
May 15th, 2012 9:42am

Hi Fejesjoco,

Yes the Oracle VSS writer is used by DPM, from the Oracle logs as well as the windows logs everything points to the Oracle VSS writer coming into use.

On the second question, yes! It does do an express full, so it is very fast transferring only the changed blocks.

During my testing I found that when the database was under HEAVY load (proportional to the IO capability/RAM/CPU of the VM) the Oracle VSS writer would fail in creating the snapshot as it could not quiesce all the databases within the given time.

I haven't had the time or large amount of hardware to do 'production load/hardware' testing, so I would be interested to see how other people go backing up Oracle in a heavy dev or production environment with DPM2012.

One downside of using the Oracle VSS writer is that it doesn't appear you have control over which tablespaces are backed up, it's an all or nothing method.

With MSSQL server they suggest not backing up more than 10DBs at the time due to quiescing timeing issues, and a hard limit of 50. So equally with the Oracle VSS writer you'd have to look at the number of tablespaces/activity and if you can quiesce in time given your performance capability of the machine.

I did notice when I was smashing the Oracle DB with inserts at the same time as trying to perform a backup, that the small VM's RAM was exhausted as the Oracle VSS writer service was taking up to 750MB of RAM, so that may have been a cause for the failed backup as well. Depending on the behavior other people see on their DPM with Oracle VSS writer they may need to set aside RAM for the Oracle VSS writer.

So people see the steps for setting this up, can you bump this up as an answer?

Install VSS writer into the desired SID ie. C:\>oravssw TESTDB /i /user:.\oracle /password:********

  1. use C:\>vssadmin list writers
  2. copy the writer ID listed against your SID
  3. Set-DPMGlobalProperty -RegisteredWriters @('VSSWriterID') -DPMservername DPMServerName
  4. Create a protection group, select the Oracle VSS Writer - SID and the object under it
  5. Watch Microsoft backup an Oracle product with a grin on your face

When you do a restore for the source I can see the following items:

  • All tablespaces (with granular DBF restore)
  • Control File
  • Flash Recovery Area
  • Server Parameter File

Kind Regards,

David

  • Proposed as answer by DBR-9 Tuesday, May 15, 2012 9:42 AM
  • Marked as answer by fejesjoco Tuesday, May 15, 2012 9:55 AM
Free Windows Admin Tool Kit Click here and download it now
May 15th, 2012 9:42am

Thanks a lot, you've been very helpful! And of course, DPM 2012 is awesome.
  • Edited by fejesjoco Tuesday, May 15, 2012 9:56 AM
May 15th, 2012 9:56am

Thanks a lot, you've been very helpful! And of course, DPM 2012 is awesome.
  • Edited by fejesjoco Tuesday, May 15, 2012 9:56 AM
Free Windows Admin Tool Kit Click here and download it now
May 15th, 2012 9:56am

Thank you for the solution.

I found an easier way to put the writer ID into DPM:

Modify-RegisteredWriters -Add -Writers "ID"

However I have come across an issue and I wondered if you had any experience of this.  On our Oracle DB server we have multiple SIDs so I proceeded to install the VSS writer for each of them.  This created a bunch of writers each with the same ID but a different Instance ID.  It appears that DPM only can see the writer ID and therefore when you kick off a job it fails instantly due to duplicate writers.

When I return the setup to only one VSS on the Oracle Server and only backup the one SID from DPM everything is fine, but this leaves us with no backup for the other SIDs.

Anyone have any thoughts about this, maybe a way to manipulate the VSS writer ID for the other SIDs?

June 21st, 2012 11:51am

DBR-9

Thanks for the info its been great, we are just testing this and have found that if i do a restore to a network location rather than the origional destination then i only seem to get some .DBF files in the database directory and not the .CTL control file file or any of the redo logs ? have you seen this ?

Cheers in advance


Free Windows Admin Tool Kit Click here and download it now
February 6th, 2013 11:25am

DBR-9

Thanks for the info its been great, we are just testing this and have found that if i do a restore to a network location rather than the origional destination then i only seem to get some .DBF files in the database directory and not the .CTL control file file or any of the redo logs ? have you seen this ?

Cheers in advance


February 6th, 2013 11:25am

Great source, thanks for the great info. I'm just curious if the oracle database can be in the NOARCHIVELOG mode or if it really must be in the other mode.

Can you please clarify? 

Thanks!

Free Windows Admin Tool Kit Click here and download it now
August 1st, 2013 8:55pm

Hello,

Actually got more into this matter with DPM 2012 SP1 ?

Thanks.

August 1st, 2013 8:58pm

Hi,

I really don't know if it will work, but if you configure diferent protection groups for each SID starting at diferent timespaces will it still give na error?

The backup probably fails because of na attempt of concurrent vss snapshots.


  • Edited by evolvtyon Friday, August 02, 2013 9:05 AM
Free Windows Admin Tool Kit Click here and download it now
August 2nd, 2013 9:05am

Hi,

I really don't know if it will work, but if you configure diferent protection groups for each SID starting at diferent timespaces will it still give na error?

The backup probably fails because of na attempt of concurrent vss snapshots.


  • Edited by evolvtyon Friday, August 02, 2013 9:05 AM
August 2nd, 2013 9:05am

DBR-9,

Did you implement this approach into a production scenario?  Did you find it workable?

thanks!

Martin

Free Windows Admin Tool Kit Click here and download it now
July 17th, 2015 5:50pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics