Showing posts with label dd. Show all posts
Showing posts with label dd. Show all posts

Tuesday, February 4, 2014

Raspberry Pi Journal #48


SD Back up time


How long does it take to back up 16 Gig SD card, anyway? Using my PNY SD 16 Gig, this is the time that it takes to fully back up the card using noob clone.

  • real 113m8.210s
  • user 2m21.400s
  • sys 24m42.570s

Or about 140 minutes. That's pretty fast.


Tuesday, January 28, 2014

Raspberry Pi Journal #47


SD card capacity


So I was wondering why my dd command always fail with insufficient disk space message. Turns out that there are different sizes with different SD card brands. Here is my size of PNY 16 gig card:


  1. sudo fdisk -l /dev/sda



  • Disk /dev/sda: 16.0 GB, 16012804096 bytes
  • 255 heads, 63 sectors/track, 1946 cylinders, total 31275008 sectors
  • Units = sectors of 1 * 512 = 512 bytes
  • Sector size (logical/physical): 512 bytes / 512 bytes
  • I/O size (minimum/optimal): 512 bytes / 512 bytes
  • Disk identifier: 0x00000000

  •    Device Boot      Start         End      Blocks   Id  System
  • /dev/sda1            8192    31275007    15633408    c  W95 FAT32 (LBA)



And here is my size of generic store brand 16 Gig card:


  • sudo fdisk -l /dev/sda
  • Warning: ignoring extra data in partition table 5

  • Disk /dev/sda: 16.0 GB, 16009658368 bytes
  • 4 heads, 16 sectors/track, 488576 cylinders, total 31268864 sectors
  • Units = sectors of 1 * 512 = 512 bytes
  • Sector size (logical/physical): 512 bytes / 512 bytes
  • I/O size (minimum/optimal): 512 bytes / 512 bytes
  • Disk identifier: 0x00048a81

  •    Device Boot      Start         End      Blocks   Id  System
  • /dev/sda1            2048     2324218     1161085+   e  W95 FAT16 (LBA)
  • /dev/sda2         2326528    31268863    14471168   85  Linux extended
  • /dev/sda5         2334720     2449407       57344    c  W95 FAT32 (LBA)



As you can see, the PNY SD card has holds 16012804096 bytes, compared to 16009658368. So that's the difference of 3145728 bytes. No wonder the dd command always fails. Well, there are two ways about it:


  1. Buy identical PNY SD cards
  2. Migrate the whole thing to a smaller, generic SD card.


Option one is easy, but depends on availability of such card. Option two is, frankly, a pain at this point. I have too many stuff in there already.

Option 3, which is to built another card, and copy the files between them somehow does not work because the keyboard stopped working. No, I don't know why, but that is why it's off the table.

Wednesday, August 14, 2013

Raspberry Pi Journal #5



SD Back Up Frustrations


So, I was going to back up my SD card. I think that there is a good chance that my SD card will go bust someday. I'd like to be protected. Considering that I install a lot of program, and it's not over yet, there is an excellent chance that the SD card will go out eventually. Especially since omxplayer is still messing up my system. The last time I used it, the screen actually got shifted in the middle. That is, all four corners of the desktop are centered on the screen. Sigh.

So, having the need to back up my SD card, how do I do it? A lot of instructions on the Internet says to use Win32 Disk Imager. I suppose it works fine, but come on, Debian is a very powerful OS. Are you saying I have to use Windows just to back up a Linux OS? That's silly.

So, I dig deeper. It turns out that I have to dig quite deep to uncover DD. DiskDump (or Disk Destroyer) seems to work fine. This is how you use it:

dd if=/dev/sdb of=/path/to/backupfile.img
if=input file or sd card
of=output file

And this is where I stop. The method obviously works if you're copying an SD card to another place, such as a flash drive. But, how will this back up my current system? I have to take it out of the machine and back it up manually? That's not how back up supposed to work, does it?

I understand that the output is not compressed, but I don't see it as a drawback. You can simply output to sdout and pipe it to gzip. There you go, compressed on the fly!

I also understand that if I specify the wrong SD card, I can mess things up. I read very carefully the text I have, but there is no specific mention how the SD card will be destroyed. I'm assuming that the destruction will happen on the output side, not the input side. But if so, isn't that natural?

All I can tell you is that the process will not backup the current running OS. Also, there is no instruction on how to restore the data. The only hint is that if "of" is given drive, it will try to "rewrite the whole disk". I'm wondering if that means the whole partition is intact. If indeed we're copying two SD cards, mirroring one another, I think that's a desirable outcome since that way, we get to skip a step. If one card is busted, well, no need to reflash the SD. Just use the back up SD right away!

Researching this method of back up is very frustrating and time consuming. Mirroring SD card is not what I'm looking for! I want to back up the OS, and I want to know how to do it, live.

I'm thinking to partition /boot and /home separately. I think I can just tar /home like this:

tar -cvzf tarball.tar /home/ 

and that's that. Extract the files by

tar -xvzf tarball.tar /home/

That should do nicely. I also wonder if the command

tar -xvf tarball.tar

all by itself is enough. I checked it. Sure enough, it works! Apparently, there is a check in there if the archive is compressed, it will automatically decompress. I checked out the output and there seems to be 4 bytes difference. I wonder why that is. There's no other differences that I can see.


There are two commands that you can use: "tar --help" and "tar --usage". Both screens are simultaneously overkill and not enough sample usage for everyday use. Hmmmph.

In any case, I still don't know how to back up my system, without shutting it down and take it out to another computer. Very frustrating!

Do you people are actually satisfied with the back up instruction that requires you to shutdown your Raspberry Pi, remove the SD card to Windows machine, and run a disk imager? I know that Linux has a live-back up system. How about doing instruction for that? I have an SD card in Raspi, and another one via USB. I want to backup the system to that USB+SD card. It's okay if I have to get a bigger card, but please don't tell me I need to shut the system down! That's just so inconvenient, and isn't the idea of getting a computer is convenience?

There are many, many web pages that tells you to use Windows to back up your SD card. Hello? Do you not see why that is not desirable? Worse, the instructions just mirror one another. I get repetitive instructions that is useless, and really, just a bunch of noise. I ran out of time trying to dig out the proper instruction. And before you ask, yes, I searched for "live backup". The few forum pages that I've seen, skirted the issue and did not answer at all.

I'm pretty sure there is a proper instruction in there somewhere, because I'm having trouble believing that people are willing to put up with shutting down the device every time you want to make a back up. It's just that the bad instructions are everywhere, and not enough people putting out good instruction to counter it. I have yet to check this, but I'm getting very frustrated. I wonder if the "back up your Pi with Windows" instructions are even in TheMagPi magazine? Because, IMHO, that's the boneheaded way to do it. Learning computers are fragile, and what good would it do to send cheap computers if you need an expensive one just to back up your data?

Of course, next thing I know, lots of people will be criticizing me for unwilling to pull the SD card out of the Raspi to make a daily back up. Uh-huh. And they wonder why people don't back up their SD cards.


BTW, I finally did an "rsync -r --progress * /media/5185-5397" where the number is the SD card on my USB. Works fine ... for a directory.