Backup mechanism(s)

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Backup mechanism(s)

HughG_TMVSE

Hi all,

 

I have a few questions about backing up Artifactory.

 

My main question is, what is/are the recommended or supported mechanism/s, on Windows Server?  I see http://www.jfrog.com/confluence/display/RTF2X/Managing+Backups (we're still on 2.6.4) describes the internal backup mechanism, which is what we're using, by default.  However, our system is hosted on a VM, and our IT guys use a combination of "Symantec Backup Exec 2010" and "Veeam Backup and Replication" to do a nightly clone of the VM to a separate SAN in a physically separate location.  I'm assuming that, if that VM clone ran while Artifactory was doing its own internal backup, I might end up with inconsistencies between metadata and filesystem, or possibly even outright corruption of either/both.  I suggested doing a scheduled shut-down of the service, running the backup, and restarting the service, but there are a couple of problems with that.

·         First, the systems we have don't let us guarantee that the backup won't run until the service has successfully shut down (and similar problems on service restart).

·         Second, we expect to soon have other sites in two different time zones, with remote repos pointing to our locals, so we don't want to shut down during our night, when they're at work!

 

A related question is, does the internal backup mechanism back up absolutely everything you need to re-build your Artifactory system?  I'm assuming it backs up all files and metadata but, e.g., does it back up the config descriptor and security descriptor?  What about the Jetty config?  Anything else I'd need?

 

Lastly, is there a good way to test the backups, beyond "fetch the most recently published artifacts and check the hashes"?

 

Thanks for any advice you can give,

 

Hugh Greene, Senior Software Developer
Toshiba Medical Visualization Systems Europe, Ltd
Bonnington Bond, 2 Anderson Place, Edinburgh EH6 5NP, UK
Tel + 44 (0)131 472 4792 / Fax + 44 (0) 131 472 4799
http://www.tmvse.com / [hidden email]

 

DISCLAIMER
Unless indicated otherwise, the information contained in this message is privileged and confidential, and is intended only for the use of the addressee(s) named above and others who have been specifically authorized to receive it. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message and/or attachments is strictly prohibited. The company accepts no liability for any damage caused by any virus transmitted by this email. Furthermore, the company does not warrant a proper and complete transmission of this information, nor does it accept liability for any delays. If you have received this message in error, please contact the sender and delete the message.

 


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users
Reply | Threaded
Open this post in threaded view
|

Re: Backup mechanism(s)

sharonalin

Hi Hugh,

One of the easiest ways to backup Artifactory is to use the built-in backup feature. It allows you to backup everything (including binaries, metadata, and configuration) and it’s done in the background without the need to shut down. A clone or a snapshot of an incremental backup directory or of a completed full backup is safe to use.

The other popular method is to backup Artifactory home folder and binaries for example with rsync and the database with the database specific tool, for example mysqldump for MySQL.

Creating a clone/snapshot of Artifactory and the database while they are running might create minor discrepancies between the binaries and the files metadata. This can happen if there is write load (transaction) on the server. Corruption of the database is quite rare (especially with databases that maintain a redo log) but we cannot guarantee it will not happen.

Hope this helps.

Sharon



On Thu, Oct 10, 2013 at 4:31 PM, Greene, Hugh <[hidden email]> wrote:

Hi all,

 

I have a few questions about backing up Artifactory.

 

My main question is, what is/are the recommended or supported mechanism/s, on Windows Server?  I see http://www.jfrog.com/confluence/display/RTF2X/Managing+Backups (we're still on 2.6.4) describes the internal backup mechanism, which is what we're using, by default.  However, our system is hosted on a VM, and our IT guys use a combination of "Symantec Backup Exec 2010" and "Veeam Backup and Replication" to do a nightly clone of the VM to a separate SAN in a physically separate location.  I'm assuming that, if that VM clone ran while Artifactory was doing its own internal backup, I might end up with inconsistencies between metadata and filesystem, or possibly even outright corruption of either/both.  I suggested doing a scheduled shut-down of the service, running the backup, and restarting the service, but there are a couple of problems with that.

·         First, the systems we have don't let us guarantee that the backup won't run until the service has successfully shut down (and similar problems on service restart).

·         Second, we expect to soon have other sites in two different time zones, with remote repos pointing to our locals, so we don't want to shut down during our night, when they're at work!

 

A related question is, does the internal backup mechanism back up absolutely everything you need to re-build your Artifactory system?  I'm assuming it backs up all files and metadata but, e.g., does it back up the config descriptor and security descriptor?  What about the Jetty config?  Anything else I'd need?

 

Lastly, is there a good way to test the backups, beyond "fetch the most recently published artifacts and check the hashes"?

 

Thanks for any advice you can give,

 

Hugh Greene, Senior Software Developer
Toshiba Medical Visualization Systems Europe, Ltd
Bonnington Bond, 2 Anderson Place, Edinburgh EH6 5NP, UK
Tel <a href="tel:%2B%2044%20%280%29131%20472%204792" value="+441314724792" target="_blank">+ 44 (0)131 472 4792 / Fax <a href="tel:%2B%2044%20%280%29%20131%20472%204799" value="+441314724799" target="_blank">+ 44 (0) 131 472 4799
http://www.tmvse.com / [hidden email]

 

DISCLAIMER
Unless indicated otherwise, the information contained in this message is privileged and confidential, and is intended only for the use of the addressee(s) named above and others who have been specifically authorized to receive it. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message and/or attachments is strictly prohibited. The company accepts no liability for any damage caused by any virus transmitted by this email. Furthermore, the company does not warrant a proper and complete transmission of this information, nor does it accept liability for any delays. If you have received this message in error, please contact the sender and delete the message.

 


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users



------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users
Reply | Threaded
Open this post in threaded view
|

Re: Backup mechanism(s)

HughG_TMVSE

Hi Sharon,

 

thanks for that, that confirms my guess.  Now I'll just have to get IT to work out the appropriate scripting to sync up a remote stop/restart of the service with the clone operation.

 

Thanks,

 

Hugh Greene, Senior Software Developer
Toshiba Medical Visualization Systems Europe, Ltd
Bonnington Bond, 2 Anderson Place, Edinburgh EH6 5NP, UK
Tel + 44 (0)131 472 4792 / Fax + 44 (0) 131 472 4799
http://www.tmvse.com / [hidden email]

 

DISCLAIMER
Unless indicated otherwise, the information contained in this message is privileged and confidential, and is intended only for the use of the addressee(s) named above and others who have been specifically authorized to receive it. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message and/or attachments is strictly prohibited. The company accepts no liability for any damage caused by any virus transmitted by this email. Furthermore, the company does not warrant a proper and complete transmission of this information, nor does it accept liability for any delays. If you have received this message in error, please contact the sender and delete the message.

 

From: Sharon Alin [mailto:[hidden email]]
Sent: 14 October 2013 09:20
To: [hidden email]
Subject: Re: [Artifactory-users] Backup mechanism(s)

 

Hi Hugh,

One of the easiest ways to backup Artifactory is to use the built-in backup feature. It allows you to backup everything (including binaries, metadata, and configuration) and it’s done in the background without the need to shut down. A clone or a snapshot of an incremental backup directory or of a completed full backup is safe to use.

The other popular method is to backup Artifactory home folder and binaries for example with rsync and the database with the database specific tool, for example mysqldump for MySQL.

Creating a clone/snapshot of Artifactory and the database while they are running might create minor discrepancies between the binaries and the files metadata. This can happen if there is write load (transaction) on the server. Corruption of the database is quite rare (especially with databases that maintain a redo log) but we cannot guarantee it will not happen.

Hope this helps.

Sharon

 

On Thu, Oct 10, 2013 at 4:31 PM, Greene, Hugh <[hidden email]> wrote:

Hi all,

 

I have a few questions about backing up Artifactory.

 

My main question is, what is/are the recommended or supported mechanism/s, on Windows Server?  I see http://www.jfrog.com/confluence/display/RTF2X/Managing+Backups (we're still on 2.6.4) describes the internal backup mechanism, which is what we're using, by default.  However, our system is hosted on a VM, and our IT guys use a combination of "Symantec Backup Exec 2010" and "Veeam Backup and Replication" to do a nightly clone of the VM to a separate SAN in a physically separate location.  I'm assuming that, if that VM clone ran while Artifactory was doing its own internal backup, I might end up with inconsistencies between metadata and filesystem, or possibly even outright corruption of either/both.  I suggested doing a scheduled shut-down of the service, running the backup, and restarting the service, but there are a couple of problems with that.

·         First, the systems we have don't let us guarantee that the backup won't run until the service has successfully shut down (and similar problems on service restart).

·         Second, we expect to soon have other sites in two different time zones, with remote repos pointing to our locals, so we don't want to shut down during our night, when they're at work!

 

A related question is, does the internal backup mechanism back up absolutely everything you need to re-build your Artifactory system?  I'm assuming it backs up all files and metadata but, e.g., does it back up the config descriptor and security descriptor?  What about the Jetty config?  Anything else I'd need?

 

Lastly, is there a good way to test the backups, beyond "fetch the most recently published artifacts and check the hashes"?

 

Thanks for any advice you can give,

 

Hugh Greene, Senior Software Developer
Toshiba Medical Visualization Systems Europe, Ltd
Bonnington Bond, 2 Anderson Place, Edinburgh EH6 5NP, UK
Tel <a href="tel:%2B%2044%20%280%29131%20472%204792" target="_blank">+ 44 (0)131 472 4792 / Fax <a href="tel:%2B%2044%20%280%29%20131%20472%204799" target="_blank">+ 44 (0) 131 472 4799
http://www.tmvse.com / [hidden email]

 

DISCLAIMER
Unless indicated otherwise, the information contained in this message is privileged and confidential, and is intended only for the use of the addressee(s) named above and others who have been specifically authorized to receive it. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message and/or attachments is strictly prohibited. The company accepts no liability for any damage caused by any virus transmitted by this email. Furthermore, the company does not warrant a proper and complete transmission of this information, nor does it accept liability for any delays. If you have received this message in error, please contact the sender and delete the message.

 


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users

 


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users