General NAS-Central Forums

Welcome to the NAS community
It is currently Wed Nov 22, 2017 2:50 pm

All times are UTC




Post new topic Reply to topic  [ 36 posts ]  Go to page Previous  1, 2, 3
Author Message
PostPosted: Thu May 28, 2009 1:19 am 
Offline

Joined: Fri Feb 06, 2009 7:49 pm
Posts: 16
Oh good... you found a useful spec on the block format.

(I composed the reply I posted a few minutes ago offline and posted it without realizing that you had already made some progress... good luck!)


Top
 Profile  
 
PostPosted: Thu May 28, 2009 4:40 am 
Offline

Joined: Thu May 07, 2009 12:52 pm
Posts: 19
danek wrote:
wim wrote:
you must be checking that i'm paying attention.. first you say we expect an XFSB at 499592200192 in the image, then you say to subtract the size from 49959200192, then you give 49949200192 in the command! which of these three numbers did you mean? ;)

The "49949200192" in the command I gave was part of "49949200192-size_of_your_disk_img". The "-" was meant to be a minus sign. This is in line with the "subtract the size" instruction. When I said we expect to see it at that offset, I meant that offset in the final reconstructed image (which we don't have yet; we need to know what we're expecting so that we can properly reconstruct the image.) So, to answer your question, I meant all three of those things. :)

actually, what i was pointing out was that 499592200192 != 49959200192 !=49949200192

thanks for the clarification re the head sector cylinder stuff.. that is quite bizarre system, how did it get so convoluted? and if it is just specifying a location on the disk, can't the same information just be given as a block number, err, LBA?


Top
 Profile  
 
PostPosted: Thu May 28, 2009 5:22 am 
Offline

Joined: Thu May 07, 2009 12:52 pm
Posts: 19
all done! i have recovered all of the data, 704 396 349 070 bytes worth, and nothing seems to be lost or corrupted, and none of it is in lost and found :D

here's what i did, the primary superblock which was found at 0x3D487A00, at the top of the first allocation group, has a number contained in it, sb_agblocks, which specifies the number of blocks between allocation groups. given the blocksize (4096 bytes) and sb_agblocks=7623172, you can work out the distance to the next superblock (0x745204000). this information was detailed in the documentation pdf file above.

so on disk 1, there existed more superblocks at the top of each allocation group:

0x003D487A00 <-- primary superblock
0x078268BA00
0x0EC788FA00
0x160CA93A00
...
0x6D4A2C3A00

that's 16 AG in total on disk 1. so the next one is expected to be found at 0x748F4C7A00, but the disk only goes to 0x7470C05FFF. therefore the next XFSB should be 0x748F4C7A00 - (0x7470C05FFF + 1) = 0x1E8C1A00 into the second disk. but it's not. as was found earlier, it was at 0x1EB80000. this means that if you concatenate the data part of disk 1 with all of disk 2, you end up with 0x1EB80000 - 0x1E8C1A00 = 0x2BE600 = 5619b of 'float' between superblocks 16 and 17 that shouldn't be there.

anyway, on disk 2 we get more evenly spaced superblocks as before, at
0x001EB80000
0x0763D84000
...
0x6D2B9BC000 <-- last SB

so 16 AGs on disk 2 aswell, making 32 AGs in total. there is more data at and after 0x7470BC0000 , i think it's part of XFS journalling system and stuff, so i think it is still necessary to image beyond the last AG.

now the question is how to trim the 5619 blocks of fat out of the image so as not to corrupt the filesystem? it was not clear to me whether i should be discarding the part before the 17th AG on disk 2, or the part after the 16th AG on disk 1. i decided to use an input offset to disk 2, which is the first option, since it was all zeros on the disk up until the superblock anyway. it is true that windows disk management thing put some rubbish into the end of disk 1, something about partitions i think, which does not belong in the image file. however as long as the AG free block, AG inode B+tree info, and AG internal free list was not told about this stuff, then that part of the disk is as good as free space (or in the worst case , a few kb of corrupted data). the other thing that was unclear was when to stop reading from disk 2. i simply imaged until the end of the disk, hoping that if the image file had extra 'stuff' on the end it would not break the filesystem, which turned out to be true.

EBD disk 1 was sdd
EBD disk 2 was sdc

Code:
ddrescue /dev/sdd /wd/xfs.img /wd/xfs1.log -i 2008125b -o 0
ls -l /wd/xfs.img
ddrescue /dev/sdc /wd/xfs.img /wd/xfs2.log -i 5619b -o 499079702016
losetup /dev/loop1 /wd/xfs.img
mkdir /ebd; mount -o ro -t xfs /dev/loop1 /ebd
cp -R -v /ebd /wd

and the cp took many hours, but that did the trick! mount didn't complain about the structure, xfs_repair wasn't needed at all, and everything seems to be in there as far as i can tell, so it looks like the image was a valid one despite a couple of wrinkles.
by the way, i was able to get a 10x speedup from ddrescue by formatting the drive /wd beforehand(!). i did not want to delete the files partially recovered from a broken image file previously, so my disk was half full already, but when i saw ddrescue getting a rate of only around 5000kB/s it would have taken something like 50 hours to finish.. i guessed it was because continuous free space was not available , so other files had to be shuffled around at the same time? so i killed it, rebooted, formatted the drive and ran ddrescue again to get 65000kB/s which only took a few hours, thats more like it.


Top
 Profile  
 
PostPosted: Thu May 28, 2009 1:09 pm 
Offline

Joined: Fri Feb 06, 2009 7:49 pm
Posts: 16
wim wrote:
actually, what i was pointing out was that 499592200192 != 49959200192 !=49949200192

What do you mean those numbers aren't the same? :) Sometimes I type funny late at night. Especially when it comes to numbers on a laptop keyboard... how I long for a numeric kepad sometimes, but that would just make the laptop too big.
thanks for the clarification re the head sector cylinder stuff.. that is quite bizarre system, how did it get so convoluted? and if it is just specifying a location on the disk, can't the same information
Quote:
just be given as a block number, err, LBA?

It's not all that convoluted when you consider that it was designed for intel processors, which naturally process data in little endian order, and that the partition table and boot code all had to fit into the single 512 byte block. If they didn't squeeze the 10-bit number and the 6-bit number into two bytes, then they would need a whole three bytes for each set of values, two each for four partitions. This would increase the size of the partition table by a whopping eight bytes, which would probably be robbed from the code area. As for why it's done as heads, cylinders, and sectors instead of just block numbers, I could be wrong, but I think it has something to do with hard drives abstracting less about their workings back in the day, which meant the physical arrangement of the data on the drive mattered a lot more.
Quote:
now the question is how to trim the 5619 blocks of fat out of the image so as not to corrupt the filesystem? it was not clear to me whether i should be discarding the part before the 17th AG on disk 2, or the part after the 16th AG on disk 1. i decided to use an input offset to disk 2, which is the first option, since it was all zeros on the disk up until the superblock anyway. it is true that windows disk management thing put some rubbish into the end of disk 1, something about partitions i think, which does not belong in the image file. however as long as the AG free block, AG inode B+tree info, and AG internal free list was not told about this stuff, then that part of the disk is as good as free space (or in the worst case , a few kb of corrupted data). the other thing that was unclear was when to stop reading from disk 2. i simply imaged until the end of the disk, hoping that if the image file had extra 'stuff' on the end it would not break the filesystem, which turned out to be true.

I'm glad you were able to get everything! I ended up coming to the same conclusion as you (that it would be best to trim the extra 5619 blocks out of the beginning of the second disk; if you look at the last set of commands I posted you'll see that I use the same numbers as you did, with the exception of limiting the end of the disk, which turned out to not be necessary...) -- I figured that the beginning of the second disk would be best for the same reasons you did, namely, that you know that it's all zeroes over there, with the exception of block 1. In reality, it's probably better to take it off the end of the first disk, as it's probably the spanned area of the disks not going all the way to the end of each disk that resulted in the extra blocks, but if it's all zeroes anyway, the result should be the same.

Anyway, congratulations, and please make sure to back up from now on!


Top
 Profile  
 
PostPosted: Tue Jun 02, 2009 4:51 pm 
Offline

Joined: Thu May 07, 2009 12:52 pm
Posts: 19
it's great to have everything back! especially my 200 gigs of music, i was not looking forward to ripping all my cds again it was a huge job. here's an old family photo from the recovered data! most of those were destroyed in a fire some years ago. thanks for all your help daniel, i am very grateful. i have made a donation to the author of HxD, because that was really useful, but i probably would have mucked it up without your advice.. if there is any way i could return the favour let me know. the whole worry of lost data was very stressful for those couple of weeks, but actually getting it solved in the end was fairly satisfying and i have learnt a lot.

i will look look into backup options .. just as soon as .. i .. have .. the .. time..... :lol:


Top
 Profile  
 
PostPosted: Mon Jun 07, 2010 8:17 am 
Offline

Joined: Thu May 07, 2009 12:52 pm
Posts: 19
just got around to backing up my data , 1 year later :lol:


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 36 posts ]  Go to page Previous  1, 2, 3

All times are UTC


Who is online

Users browsing this forum: No registered users and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group