Although I have been using SD cards for a long time in many different
ways, among other things in digital cameras, until last week or so I
never thought of copying all the data from a card in one go, when
the card is getting full. Even after a hike or a trip and taking some
pictures I just copied the most recent ones and when I the card filled
I simply reformatted it, knowing that I had already saved earlier all
the files I cared about.
This time I tried to get everything and make a tar file of it.
Something like this:
mount /dev/sdb1 mnt
tar cf - -C mnt . | gzip -9 - > sdcard.tar.gz
And I got a rude surprise: sooner or later syslog spat out a message
like this one below:
Feb 8 15:55:24 matica kernel: [27666.861621] blk_update_request: critical medium error, dev sdb, sector 91120 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
and then the tar process exited unsuccessfully, complaining about a
read error.
First I suspected the card was dying — I have many quite old 2G ones.
I tried the same thing with a different card, same result! Also,
interestingly, the sector number where the error happened was
different each time even with the same card and nothing else changed.
And reading the entire filesystem image from the card with dd didn’t
help either — now it was dd dying instead of tar.
Finally I concluded that the problem was overwhelming the card with
read operations too quickly, and not giving it a break when it made
clear it needed one, by responding slowly.
Fortunately I remembered a program that had been written to solve
precisely this kind of problem, namely GNU ddrescue. It has
options to bound the read rate, to pause after slow reads, and so on.
I wrote a script to automate the whole operation, using the Linux loop
mount feature to keep the data during an intermediate step:
umask 0077
OUTFILE=/dev/stdout
case "${1:-}" in
(-o) OUTFILE="${2:-/dev/stdout}" ; shift 2 ;;
esac
INPUTDEV=${1:-/dev/sdb1}
JOBDIR=$(mktemp -d /var/tmp/tarcard.XXXXXXXXXXXX)
printf '%s\n' $JOBDIR >&2
IMGFILE=$JOBDIR/image
MAPFILE=$JOBDIR/map
EVENT_LOGFILE=$JOBDIR/eventlog
RATES_LOGFILE=$JOBDIR/rateslog
READS_LOGFILE=$JOBDIR/readslog
MOUNT_POINT=$JOBDIR/mnt
ddrescue -q -Z 4M -c 4096 -X 0 --pause-on-error=3s \
--log-events=$EVENT_LOGFILE \
--log-rates=$RATES_LOGFILE \
--log-reads=$READS_LOGFILE \
"$INPUTDEV" $IMGFILE $MAPFILE
LOOPDEV=$(losetup --show -P -r -f $IMGFILE)
mkdir $MOUNT_POINT
mount -r $LOOPDEV $MOUNT_POINT
tar cf "$OUTFILE" -C $MOUNT_POINT .
umount $MOUNT_POINT
rmdir $MOUNT_POINT
losetup -d $LOOPDEV
rm -rf $JOBDIR
So far I have been able to read the card each time — about 5 times so
far. I’ll update this post if I get disappointed yet …