fsarchiver forums
View unanswered posts | View active topics It is currently Tue Jul 29, 2014 4:35 am



Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next
 MBR & VBR backup support 
Author Message

Joined: Tue Jan 12, 2010 3:38 pm
Posts: 9
Post MBR & VBR backup support
Reiterating what I added to one of the old posts in General Questions, I think in order to be a functional replacement for partimage, fsarchiver should handle a few more things. Namely:

1) Should handle FAT (this is still used, particularly on removable devices)
2) Should backup and restore the partition's VBR such that Windows is bootable after restoration.
3) Should backup and restore the MBR

As for the MBR, it is very trivial to implement and requires only 512 bytes of storage space. I suggest having fsarchiver save all MBRs of all relevant drives in the archive by default. Then give the user a few options for restoring the MBR (eg full MBR or just the boot code). This MBR backup script is the kind of functionality I mean.

By backing up and restoring the VBRs by default, and by backing up the MBR by default and giving the user the option to restore it, fsarchiver becomes a complete backup solution for restoring the partitions and the boot process. Otherwise it is an incomplete backup solution even for linux, and is not a functional replacement for partimage, particularly for Windows partitions.


Tue Jan 12, 2010 4:15 pm
Profile

Joined: Tue Jan 12, 2010 3:38 pm
Posts: 9
Post Re: MBR & VBR backup support
I would like to rewrite my wikibook - How To Backup Operating Systems, so you might review that for an idea of the kind of functionality I mean. I would like to rewrite that using one tool such as fsarchiver, but at present the best I can do is use partimage for Windows partitions, fsarchiver for linux partitions, and dd/sfdisk or the mbrback script for MBR and partition table backup. I think fsarchiver could do it all without getting too far from what it already is.


Tue Jan 12, 2010 4:31 pm
Profile

Joined: Tue Jan 12, 2010 3:38 pm
Posts: 9
Post Re: MBR & VBR backup support
I just came across this...
http://www.fsarchiver.org/Cloning-ntfs

Does this mean that fsarchiver does save the VBR and/or other Windows boot info on the partition? (Something I read elsewhere led me to believe otherwise.) For example, can I backup a Windows ntfs partition on partition 1 and save it to a completely blank partition 2 (no windows VBR boot info already on it), and expect it to boot? (with me changing the boot partition or grub to reflect the new location of course). Thanks.

If so, then the only features I suggest adding are FAT support and MBR backup/restore. (Full partition table backup and restoration like sfdisk does would be nice too, but not necessary as sfdisk already does this.)


Tue Jan 12, 2010 5:06 pm
Profile
Site Admin

Joined: Sat Feb 21, 2004 12:12 pm
Posts: 544
Post Re: MBR & VBR backup support
1) Windows should still boot if you restore your ntfs to another partition, as long as you edit the boot.ini file, by mounting it using ntfs-3g and using an editor.

2) It does not support FAT filesystems because people would expect it to boot if they have a Windows 9x installed on it, and I don't know how to preserve that.

3) Backing up the MBR would be possible, but the idea is that partimage is a static tool (it works at the block level) and fsarchiver is a dynamic tool (it recreates the appropriate structure on a disk that can be different). The best thing would be to dump the partition table, and to recreate it, including the logical drives.


Tue Jan 12, 2010 8:18 pm
Profile

Joined: Tue Jan 12, 2010 3:38 pm
Posts: 9
Post Re: MBR & VBR backup support
admin wrote:
1) Windows should still boot if you restore your ntfs to another partition, as long as you edit the boot.ini file, by mounting it using ntfs-3g and using an editor.


Okay thanks.

Quote:
2) It does not support FAT filesystems because people would expect it to boot if they have a Windows 9x installed on it, and I don't know how to preserve that.


Fair enough. Maybe that is something you could look into later. I believe if you determine the size of the VBR and restore it, that would be all that is required, but I'm no expert on it. Since bootable FAT partitions are still in regular use (USB sticks and such particularly), I think this would be worth exploring.

In the meantime, this fact should be added to your partimage vs fsarchiver page, as no FAT support is a significant difference between the two.

Quote:
3) Backing up the MBR would be possible, but the idea is that partimage is a static tool (it works at the block level) and fsarchiver is a dynamic tool (it recreates the appropriate structure on a disk that can be different). The best thing would be to dump the partition table, and to recreate it, including the logical drives.


sfdisk does fine with dumping and recreating the partition table. The main use of having fsarchiver backup and restore the MBR would be to restore the boot code (which is the first 448 bytes of the 512 byte MBR). It can be restored with:
Code:
dd if=mbrbackup of=/dev/sda bs=448 count=1


That will restore just the boot code without touching the partition table. It would be nice for fsarchiver to do this because the other common system apps don't and it's a trivial addition. Maybe something like...

Code:
fsarchiver savembr /mnt/backup/mbrbackup.fsa /dev/sda


And/or, you could have a "-m <device>" option which causes fsarchiver to backup the MBR and include it in the archive with the filesystem...
Code:
fsarchiver -m /dev/sda savefs /mnt/backup/backup.fsa /dev/sda1


And then separate restore command(s):
Code:
# restores just the boot code
fsarchiver restmbrboot /mnt/backup/mbrbackup.fsa /dev/sda
# restores full MBR with primary partition table
fsarchiver restmbrfull /mnt/backup/mbrbackup.fsa /dev/sda


With the MBR addition, fsarchiver is a full system backup solution for linux/windows multi-boot system, including the boot process (at least for non-FAT partitions).

The only caveat would be that since fsarchiver moves the kernel files, the MBR boot code might need to be reinstalled anyway (although this is not supposed to be the case with grub). But it would still provide an option for backing up and restoring the MBR in some situations, such as viruses, installing Windows after linux and wanting to put the boot code back, saving the Windows boot code before replacing it with grub, etc. How it's used is up to the user, but I think it would be nice to have the ability in there.


Tue Jan 12, 2010 8:52 pm
Profile
Site Admin

Joined: Sat Feb 21, 2004 12:12 pm
Posts: 544
Post Re: MBR & VBR backup support
1) Support for FAT
About the FAT support: if you can find documentation that says how to preserve the stuff required to boot Windows, or if you can demonstrate how to do that by hand, I can consider adding support for FAT.

For your tests, you need two partitions. One original that you are trying to clone, and an empty partition for your tests. They both have to be primary partitions. Basically the filesystem will be recreated using mkdosfs. You can copy the files between /mnt/part1 and /mnt/part2 using rsync. After that, what do we have to do ? Do we have to copy boot code in the boot sector, or in another sector ? Do we have to update the reference to a block in the boot sector ?

2) Backup and recreation of the partition table
I am thinking about how to implement a backup/recreation of the partition table. It should really be possible, and we can use libparted for that. It has to support both msdos and gpt disklabels. This way it can dump extended/logical partitions as well as primary.It also provides a support for disklabels other than msdos, like gpt which can be used on normal PCs. I really prefer doing that dynamically (we understand what we copy) instead of statically (we copy a sector), because it will provide more flexibility.

About the boot code which is stored in the MBR.
a) It should not be a problem for disks with just Windows installed. We can just recreate a standard MBR with a partition table, restore the ntfs filesystem. The BIOS will just boot the active partition, no particular action is required.
b) For linux based disks, we can reinstall grub. fsarchiver can have a "--reinstall-grub" command that will chroot in the root filesystem and execute "grub-install /dev/xxx". Fortunately the big majority of Linux Operating Systems are now using grub and not lilo, so we just have to provide support for Grub.


Sat Jan 16, 2010 11:15 am
Profile

Joined: Tue Jan 12, 2010 3:38 pm
Posts: 9
Post Re: MBR & VBR backup support
admin wrote:
1) Support for FAT
About the FAT support: if you can find documentation that says how to preserve the stuff required to boot Windows, or if you can demonstrate how to do that by hand, I can consider adding support for FAT.

For your tests, you need two partitions. One original that you are trying to clone, and an empty partition for your tests. They both have to be primary partitions. Basically the filesystem will be recreated using mkdosfs. You can copy the files between /mnt/part1 and /mnt/part2 using rsync. After that, what do we have to do ? Do we have to copy boot code in the boot sector, or in another sector ? Do we have to update the reference to a block in the boot sector


From the last I researched this topic, Windows uses the VBR (volume boot record aka partition boot record) to store its boot code. Unlike the MBR though, the VBR can vary in size, but I believe it is always located at the start of the partition.

Lots of good info here...
http://mirror.href.com/thestarman/asm/mbr/

Here is Win95-Me with FAT: "Boot Record is actually 3 sectors long, and is found at Logical Sectors 0 through 2".
http://mirror.href.com/thestarman/asm/mbr/MSWIN41.htm

WinXP with FAT32
http://mirror.href.com/thestarman/asm/mbr/ntFAT32BR.htm
Which also mentions: "Reminder: Don't forget that each FAT32 Boot Record has a Backup Copy (even under Windows 2000 or XP) just a few sectors beyond the original. These correspond to Relative (or Logical) Sectors 6 through 8 (for the Backup copy) of any FAT32 partition on your drive."

WinXP with NTFS
http://mirror.href.com/thestarman/asm/mbr/NTFSBR.htm

Here is a breakdown of Vista, which uses a VBR of 2048 bytes...
http://mirror.href.com/thestarman/asm/mbr/VistaVBR.htm

I think the ultimate solution would be to figure out a generic method which determines the VBR size, such that it works for any device/OS (for example a Garmin GPS unit, which partimage can handle). For the kernel or filesystem make command to know where to start the filesystem, this info must be able to be determined. That would avoid having to create different profiles for each OS that might be installed on a partition, and might even work for NTFS the same as it does for FAT. I don't think you need to care what's in the VBR, unless it has a block pointing to one of the boot files (which will change when fsarchiver recreates the fs).

I don't actually have windows installed on any FAT partitions, but if I get a chance I will see what else I can find on the subject. You might even have a look at the partimage code to see how it determines where the filesystem begins. Partimage does process the filesystem (in order to avoid copying unused blocks), so it must know where the VBR is and its size. I know it always copied the VBR successfully.
http://www.partimage.org/Partimage-manu ... .2Fwritten

Quote:
2) Backup and recreation of the partition table
I am thinking about how to implement a backup/recreation of the partition table. It should really be possible, and we can use libparted for that. It has to support both msdos and gpt disklabels. This way it can dump extended/logical partitions as well as primary.It also provides a support for disklabels other than msdos, like gpt which can be used on normal PCs. I really prefer doing that dynamically (we understand what we copy) instead of statically (we copy a sector), because it will provide more flexibility.


Agreed. That is how sfdisk works, which works well. It outputs a plain text file which is both human- and machine-readable. It can then accept that file to recreate the table. I actually don't know if you need to get into that, as sfdisk does it already, except that it would be nice to have it in one program/archive file. In fact maybe you could borrow some code from sfdisk.

Quote:
About the boot code which is stored in the MBR.
a) It should not be a problem for disks with just Windows installed. We can just recreate a standard MBR with a partition table, restore the ntfs filesystem. The BIOS will just boot the active partition, no particular action is required.
b) For linux based disks, we can reinstall grub. fsarchiver can have a "--reinstall-grub" command that will chroot in the root filesystem and execute "grub-install /dev/xxx". Fortunately the big majority of Linux Operating Systems are now using grub and not lilo, so we just have to provide support for Grub.


That might be a good approach, providing you can Keep It Simple. Grub sometimes can get very unsimple depending on multiple OSes on the system, and also the complexities of grub v1 vs v2 now. BTW, on SystemRescueCD, grub2-install is v2, and grub-install is v1. (This isn't documented anywhere that I could find.)

Also, I looked into it and the way fsarchiver recreates the filesystem and presumably moves the sector locations of the files, grub WILL need to be reinstalled to the MBR afterward (rather than simply copying the boot code). Grub stores the disk sector of its stage2 file in the MBR boot code, so if that file is moved to a different location on disk, grub will most likely report a file not found error.

So given the fact that fsarchiver is going to break grub when it restores a partition containing grub's files, addressing grub reinstallation in some way is probably a good idea. At the very least, fsarchiver's docs should address it. (Note that with the partimage method, the files are not moved, so grub doesn't break.)

I have updated my wikibook with fsarchiver methods, including the grub breakage issue. Since fsarchiver can't completely replace partimage yet, I wrote it as a dual-method approach, but I made fsarchiver the recommended method for linux filesystems. I recommended against using fsarchiver for NTFS at present because you said elsewhere it wasn't as stable, but I can update that as fsarchiver evolves.
http://en.wikibooks.org/wiki/How_To_Bac ... ng_Systems

Overall I really like fsarchiver's design. I think restoring to smaller partitions than the original and other features is worth the grub breakage.


Sat Jan 16, 2010 3:39 pm
Profile

Joined: Wed Dec 16, 2009 8:18 pm
Posts: 18
Post Re: MBR & VBR backup support
additionally, as previously mentioned fsarchiver needs a way to verify the image before it can replace partimage. I am anxiously awaiting that feature.


Tue Jan 19, 2010 6:54 pm
Profile
Site Admin

Joined: Sat Feb 21, 2004 12:12 pm
Posts: 544
Post Re: MBR & VBR backup support
I am currently working on backup and recreation of the partition table (one thing at a time), and it seems to be working quite well. The complicated problems have been addressed, especially respecting partition numbers when numbers are missing such as a disk with sda1 sda3 sda3 but no sda2. Some stuff is still missing (preservation of partition flags) but it should not be difficult to do that. I expect that to be ready in the next few weeks. Help will be welcome to test that feature when a beta version is online.


Sun Jan 31, 2010 10:38 pm
Profile

Joined: Tue Jan 12, 2010 3:38 pm
Posts: 9
Post Re: MBR & VBR backup support
Sounds good.

I was thinking that you may want to look into updating the grub boot code with the new disk sector of its stage file. It doesn't store much in there. Maybe check to make sure the boot code is grub and is also a version that you support, then update it, or better, provide that as an option. This would save you from having to get into the thick of grub configuration issues. One of the nice things about partimage is I can restore my partition and boot - no need to reinstall grub. fsarchiver could also note the location of that file in the archive, then check the boot code for it. Anyway something to explore.

MBR code details for Gurb and Lilo are here... not sure about grub2 but you should be able to find it somewhere.
http://mirror.href.com/thestarman/asm/mbr/#grub


Wed Feb 03, 2010 4:45 pm
Profile
Site Admin

Joined: Sat Feb 21, 2004 12:12 pm
Posts: 544
Post Re: MBR & VBR backup support
It would be ideal if grub could be reinstalled with no dependency on any grub program. But it's really complicated with grub1+grub2 and stage1.5 + stage2. Running grub-install in a chroot may be the only option.

Anyway I am currently working on the support for dump/restore of partition tables. I think all the features are implemented now. I have to make a big cleanup in the code, to integrate the stuff with the other command line options, and to make many tests. I will publish a beta1 version as soon as it's ready.


Wed Feb 03, 2010 10:10 pm
Profile
Site Admin

Joined: Sat Feb 21, 2004 12:12 pm
Posts: 544
Post Re: MBR & VBR backup support
I have released fsarchiver-0.7.0_beta1.tar.gz, which add support for savept, restpt, showpt (save and restore partition table).
This is based on libparted, then parted-devel / parted-dev required to compile fsarchiver (as well as libxml2). It has been tested on libparted-1.8.8 and libparted-2.1.

Feed back is welcome. Be careful, this is a beta version. Also "restpt" can be dangerous if you restore a partition table on the wrong disk. It's recommended to make tests on USB sticks or in virtual machines.

The partition tables of all disks are automatically saved when we do a "savefs" except if "--nosavept" is used. You can also save the partition table with no filesystem in the archive, using "savept" instead of "savefs". You can use "showpt" to see the partition tables (similar to archinfo). Be very careful with "restpt": this is a dangerous command. It has the same syntax as "restfs".

There will be changes in next beta versions, so please don't assume that an archive created with fsarchiver-0.7.0-beta1 will work with other versions. It may be necessary to add missing attributes, to change names, or that sort of things.

The current version should support the following features:
1) it supports both the "msdos" and "gpt" disk labels (99% of the users should be happy)
2) it works for primary partitions (aka normal) as well as extended and logical
3) it should preserve the partition attributes (size, position, numbers, flags)
4) partitions orders should be preserved (partition 1 may be at the end of the disk)
5) partition numbers should be preserved even if numbers are missing (eg: sda1, sda2, sda4 but no sda3)
6) it should work on devices with sector_size different from 512, but needs more testing

Code:
root@debian ~/fsarchiver-git/src % ./fsarchiver savept -o -v /var/tmp/myarchive.fsa
Saving description of the partition table of disk /dev/sda
Saving description of the partition table of disk /dev/sdb
Saving description of the partition table of disk /dev/sdc

root@debian ~/fsarchiver-git/src % ./fsarchiver archinfo /var/tmp/myarchive.fsa
====================== archive information ======================
Archive type:                   filesystems
Filesystems count:              0
Partition tables count:         3
Archive id:                     4b763aa9
Archive file format:            FsArCh_002
Archive created with:           0.7.0-git
Archive creation date:          2010-02-14_16-45-46
Archive label:                  <none>
Minimum fsarchiver version:     0.6.4.0
Compression level:              3 (gzip level 6)
Encryption algorithm:           none

root@debian ~/fsarchiver-git/src % ./fsarchiver showpt /var/tmp/myarchive.fsa

id=0: Original disk: [/dev/sda] [931.51 GB] [ATA ST31000340AS] (disklabel: msdos)
   -partition=01 start=00000019531 len=00002000001 end=00002019531 (normal)
   -partition=02 start=00002019532 len=00032000000 end=00034019531 (normal)
   -partition=03 start=00034019536 len=00031999872 end=00066019407 (normal)
   -partition=04 start=00066019532 len=01887505636 end=01953525167 (normal)

id=1: Original disk: [/dev/sdb] [1.82 TB] [ATA Hitachi HDS72202] (disklabel: gpt)
   -partition=01 start=00000000034 len=00000019498 end=00000019531 (normal)
   -partition=02 start=00000019532 len=00002000000 end=00002019531 (normal)
   -partition=03 start=00002019532 len=00032000000 end=00034019531 (normal)
   -partition=04 start=00034019532 len=00032000000 end=00066019531 (normal)
   -partition=05 start=00066019532 len=00032000000 end=00098019531 (normal)
   -partition=06 start=00098019532 len=00032000000 end=00130019531 (normal)
   -partition=07 start=00130019532 len=01823505859 end=01953525390 (normal)
   -partition=08 start=01953525391 len=00600000000 end=02553525390 (normal)
   -partition=09 start=02553525391 len=01024000000 end=03577525390 (normal)
   -partition=10 start=03577525391 len=00329503744 end=03907029134 (normal)

id=2: Original disk: [/dev/sdc] [931.51 GB] [ATA SAMSUNG HD103UJ] (disklabel: msdos)
   -partition=01 start=00000000063 len=00000289107 end=00000289169 (normal)
   -partition=02 start=00000289170 len=00000273105 end=00000562274 (normal)
   -partition=03 start=00000562275 len=00000321300 end=00000883574 (normal)
   -partition=04 start=00000883575 len=00000273105 end=00001156679 (normal)

root@debian ~/fsarchiver-git/src % ./fsarchiver restpt /var/tmp/myarchive.fsa id=0,dest=/dev/sdc

id=0: Original disk: [/dev/sda] [931.51 GB] [ATA ST31000340AS] (disklabel: msdos)
   -partition=01 start=00000019531 len=00002000001 end=00002019531 (normal)
   -partition=02 start=00002019532 len=00032000000 end=00034019531 (normal)
   -partition=03 start=00034019536 len=00031999872 end=00066019407 (normal)
   -partition=04 start=00066019532 len=01887505636 end=01953525167 (normal)

Destination disk: [/dev/sdc] [931.51 GB] [ATA SAMSUNG HD103UJ] (disklabel: msdos)

Partition table will be restored on /dev/sdc in 4 seconds, press Ctrl+C to abort
Partition table will be restored on /dev/sdc in 3 seconds, press Ctrl+C to abort
Partition table will be restored on /dev/sdc in 2 seconds, press Ctrl+C to abort
Partition table will be restored on /dev/sdc in 1 seconds, press Ctrl+C to abort
Partition table will be restored on /dev/sdc in 0 seconds, press Ctrl+C to abort
Restoration of the partition table on disk=[/dev/sdc]...
Partition table has been written on disk=[/dev/sdc]


Sun Feb 14, 2010 5:43 pm
Profile

Joined: Tue Jan 12, 2010 3:38 pm
Posts: 9
Post Re: MBR & VBR backup support
Thanks for the update. Next time I make a backup, which will be shortly, I'll check out the saving of the table. I probably won't actually restore the partition table though. :) I don't have a spare drive to test on at the moment. Next time I clear a drive I'll try it.


Tue Feb 16, 2010 1:26 am
Profile

Joined: Mon Mar 22, 2010 6:57 pm
Posts: 4
Post Re: MBR & VBR backup support
The default in grub2 is to embed the core in the area between the MBR and the first partition on the disk. As long as you backup and restore those sectors along with the MBR, that's all you have to do for grub to continue to work, assuming that you don't move the partition with /boot/grub on it. If you do restore that to a different partition, then grub2 will drop you to a rescue shell where you can tell it the correct location of /boot/grub and load the normal module to switch to the normal mode and proceed with a normal boot.

You get similar behavior with grub1 when you use stage 1.5, except that it doesn't have the rescue mode when stage2 is moved to another partition. Without stage 1.5 ( the default behavior ) the MBR contains a direct pointer to the sectors occupied by the main grub image. To restore this correctly, you will have to update those pointers with the new address. This should be fairly straight forward to do directly without trying to call out to external tools.


Mon Mar 22, 2010 7:19 pm
Profile
Site Admin

Joined: Sat Feb 21, 2004 12:12 pm
Posts: 544
Post Re: MBR & VBR backup support
There are too many assumptions to try to preserve grub this way. Also I want fsarchiver to be a dynamic tool (it recreates things that corresponds to the new situation) and not a static tool (like partimage that just replicate blocks and which may not be appropriate on a different hardware).

Ideally I would like fsarchiver to optionally run the grub-install command. This way it would not need to worry about existing blocks and it would work if you clone filesystems on a new disk. I have to think about how to do that. It has to work on all sort of configurations (multiple linux root filesystems, multiple disks, ...)


Sun Apr 11, 2010 8:10 am
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by Vjacheslav Trushkin for Free Forums/DivisionCore.