In the best of all possible worlds, the CF cards, SD cards, and other external memory devices we buy would work every time, and we would never have to be concerned about their integrity. But, sad to say, that is not the case. Anyone using SD or CF cards on their Zaurus has probably had a problem with expansion cards at some point or other.
There are plenty of other discussions out there about reformatting, compatibility, accessing files, and transferring data to and from SD and CF cards. What I want to focus on here is how to test the integrity of a card and it's files.
While the general principles I will discuss here
probably apply to all Zauruses and other Linux systems, please note that
I am posting examples and caveats based on running
the commands using an sl5500 Zaurus with Sharp ROM 2.38,
which uses the 2.4.6 Linux Kernel, and the output
from other ROMs and kernels may vary.
In some cases, you may be unable to get a card's icon to appear in the taskbar on your Zaurus after inserting the card, and be unable to get the following commands to run as shown because the file system has not mounted (example here). Before you give up, reboot your system with the card inserted. This may clear any problems preventing you from successfully doing the suggested diagnostics, with the icon appearing and commands working as shown below after rebooting.
If you are attempting to diagnose or repair problems with expansion cards mounted over USB, you may get very misleading, discouraging output unless you eject and reinsert the media, as well as possibly rebooting your Zaurus or other system. Do not assume that if the commands listed below indicate that the file system has been corrupted, do not assume that is a correct diagnostic.
I am finding that when mounting or repairing SD cards in a USB card reader, both the Zaurus and the Raspberry Pi will sometimes indicate a broken file system when the file system is not broken. Before you assume that the file system or partition structure is beyond repair, try removing and reinserting the card in it's card reader. For the Raspberry Pi, that often suffices, for the Zaurus, I am finding that I need to reboot the Zaurus twice in a row before it can properly recognize and either mount or properly repair the file system on an expansion card mounted via USB.
The first thing we all should do, when we buy a new card or other memory device, is test it's physical integrity, as a piece of hardware.
The discussion in this section and the following one about unmounting media, was written on the assumption you are testing media that has a file system that has already been mounted on your Zaurus or other computer. If your USB stick, CF or SD card, or other memory device has not been mounted, then you will need to mount the file system in order to run the commands in this section.
As part of our testing of the physical integrity of a card, however, we need to determine what type of file system is on the card, if there is one. Confused? Huh? What difference does the file system make if I want to test the hardware, not the file system? Does that seem backwards? I understand, but just read on. I think that is part of why it has been hard for me to figure out how to do this myself, so if you feel like a nerd about it, you are not alone. ;)
We will need to use the "mount" command to find out the file system type.
And we will not give it any arguments in this case, or it will not yield the information we want.
Here is some sample output:
bash-2.05# mount
/dev/mtdblock0 on / type cramfs (ro)
/proc on /proc type proc (rw)
/dev/ram1 on /dev type minix (rw)
/dev/mtdblock1 on /home type ext2 (rw,sync)
none on /dev/pts type devpts (rw)
/dev/mmcda1 on /usr/mnt.rom/card type vfat (rw)
bash-2.05#
Now, if parsing the output of "mount" is confusing to you, just "pipe" the output through "grep" as follows. The first example below is for an SD card, and the second is for a CF card:
bash-2.05# mount | grep card
/dev/mmcda1 on /usr/mnt.rom/card type vfat (rw)
bash-2.05#
bash-2.05# mount | grep cf
/dev/hda1 on /usr/mnt.rom/cf type ext2 (rw)
bash-2.05#
The above output tells me that my SD card has a vfat file system, and that my CF card is formatted with ext2.
The next thing we will do with a card that is not new, is run the fsck command, the acronym for "file system check." But this can get weird, as we have more than one fsck command on the Zaurus! To see which fsck commands I have on my Zaurus, I use something called "tab expansion". I simply enter just the first two letters of "fsck" (i.e., "fs") on the command line, and then hit the tab three times:
bash-2.05# fsck
fsck fsck.ext3 fsck.msdos
fsck.ext2 fsck.minix fsck.vfat
bash-2.05#
So, since my SD card has a vfat file system, I see I can use fsck.vfat to check it. And, since my CF card is formatted in ext2, I should use fsck.ext2 to check it's file system.
However, in order to run fsck, we must first detach the file system by using the "umount" command. Note that while the purpose of "umount" is to unmount the file system while the card is still physically attached to your Zaurus, the command is "umount", without any "un" in it.
The discussion in this section and the following one about unmounting media, was written on the assumption you are testing media that has a file system that has already been mounted on your Zaurus or other computer. If your USB stick, CF or SD card, or other memory device has not been mounted, and you already know the file system type, then you can skip this section, and move on to the section about using fsck.
After issuing the "umount" command, you will need to
wait until the SD card icon has disappeared from
the bottom tray on your Zaurus before you enter the fsck command.
On my Sharp ROM 2.38, the display freezes a little while and then the
SD card icon disappears, as it should.
However, when I attempt to "umount" the card in my CF slot, the CF card icon remains in the bottom tray, even after the card has been unmounted. So, to confirm that the unmounting is complete, I run the "df" command.
The following is sample output from the "df" command, showing both an SD and a CF card mounted on my Zaurus:
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/ram1 44 25 19 57% /dev
/dev/mtdblock1 32619 31519 0 100% /home
/dev/mmcda1 499968 397216 102752 79% /usr/mnt.rom/card
/dev/hda1 1006688 758848 247840 75% /usr/mnt.rom/cf
The following shows my Zaurus with just an SD card mounted:
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/ram1 44 26 18 59% /dev
/dev/mtdblock1 32619 31273 0 100% /home
/dev/mmcda1 499968 390544 109424 78% /usr/mnt.rom/card
The following shows my Zaurus with just a CF card mounted:
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/ram1 44 25 19 57% /dev
/dev/mtdblock1 32619 26690 4189 86% /home
/dev/hda1 500176 16 500160 0% /usr/mnt.rom/cf
And the following shows my Zaurus without any cards mounted:
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/ram1 ; 44 26 18 59% /dev
/dev/mtdblock1 32619 32044 0 100% /home
Once a SD card has been successfully unmounted, there will be no output from "df" showing "/usr/mnt.rom/card". And, similarly, once a CF card has been unmounted. there will be no lines with information about "/usr/mnt.rom/cf".
Note that on later ROMs, which usually have the user logged in as "zaurus" rather than as "root", you may be unable to run the umount and other commands because of security issues, unless you utilize the sudo command.
In the following example, I was not able to run the umount command on my sl6000.
Here is the error message:
bash-2.05# umount /dev/mmcda1
umount: /dev/mmcda1: Operation not permitted
bash-2.05#
I had to preface the umount command with sudo,
which gave me root permissions.
I used sudo to run the umount command as follows to unmount my SD card:
bash-2.05# sudo umount /dev/mmcda1
bash-2.05#
Sudo is not necessarily built in to all Zaurus ROMs, so you many need to install it off of one of the feeds if you have a later ROM and have not already installed it.
If you do not have sudo, you will get output like the following from the type command:
bash-2.05# type sudo
bash: type: sudo: not found
bash-2.05#
If you have it installed properly, you will get a message about it's location if you run the type command for it, as follows:
bash-2.05# type sudo
sudo is hashed (/usr/bin/sudo)
bash-2.05#
Most of the commands recommended below also require root privileges but, for the sake of simplicity, I have not shown sudo in the discussion.
I will write more about sudo when I can squeeze it in to my schedule.
In the meantime, just remember that any time you get an "Operation not permitted" message, it might mean you need to preface the command with sudo in order to proceed.
The version of sudo I have gives me root privileges when I issue it, but not all installations do that.
After successfully unmounting the card, since the output of the "mount" command told me that my SD card has a vfat file system, I will want to use fsck.vfat to check the integrity of my SD cards.
Unfortunately, though, Sharp's fsck.vfat is a little buggey on ROMS 2.38 and 3.10, and you can not count on it detecting problems, not unless you give it some arguments.
For example, the following output from running plain fsck.vfat on an SD card that has been giving me trouble shows no errors. But notice there is no statement about whether or not the card is "clean::
bash-2.05# umount /mnt/card
bash-2.05# fsck.vfat /dev/mmcda1
dosfsck 2.8, 28 Feb 2001, FAT32, LFN
/dev/mmcda1: 3075 files, 24409/31248 clusters
bash-2.05#
The output from running fsck.vfat on a "clean" card should look more like the following example. Notice the word "clean" on the last line.
bash-2.05# fsck.vfat /dev/mmcda1
dosfsck 2.8, 28 Feb 2001, FAT32, LFN
/dev/mmcda1: clean, 11/125488 files, 15851/500440 clusters
bash-2.05#
Sharp's fsck will correct typical errors if there are any, but it will not give diagnostics if there are other problems
unless it is given the proper flags on the command line.
Here is the output obtained when I added appropriate options:
bash-2.05# fsck.vfat -v -t -r /dev/mmcda1
dosfsck 2.8 (28 Feb 2001)
dosfsck 2.8, 28 Feb 2001, FAT32, LFN
Boot sector contents:
System ID " "
Media byte 0xf8 (hard disk)
512 bytes per logical sector
16384 bytes per cluster
1 reserved sector
First FAT starts at byte 512 (sector 1)
2 FATs, 16 bit entries
62976 bytes per FAT (= 123 sectors)
Root directory starts at byte 126464 (sector 247)
512 root directory entries
Data area starts at byte 142848 (sector 279)
31248 data clusters (511967232 bytes)
63 sectors/track, 16 heads
233 hidden sectors
1000215 sectors total
/2005-12-14-11-31.backup
Cluster 950 (31249) is unreadable. Skipping it.
Internal error: next_cluster on bad cluster
bash-2.05#
So, if you have a vfat card, and do not see the word "clean" in the last line of the output, then I would recommend running fsck.vfat with the arguments shown in the latter example above. If you want to automatically let fsck make repairs, then substitute "-a" for "automatically repair", instead of using "-r" for "interactively repair", in the above example.
NOTE: I have seen man pages that reverse the meaning of "-a" and "-r" from what is shown in the Busybox fsck that I have on my Zaurus. So, if you want to be sure about whether repairs occur automatically as opposed to your being asked about each repair, read the relevant man pages and help files for your version of fsck, to be sure.
Now, there may be some times when it will not be necessary to use these flags. For example, I recently had forgotten to put my Collie in it's docking station when online, and was working with editing or downloading files on the SD card. All of a sudden, the Collie shut itself off due to a lack of power. That was fine, I let it charge for a little while, and was then able to use it.
However, when I attempted to write to the card, I got the disconcerting message "Read Only File System: Unable to write package.gz." I had forgotten to run fsck after the crash. But, in this case, when I did run it, I was able to run fsck without any arguments and that was enough to restore read permission to the card. What I have seen happen with both my Collie and my Tosa is that if the system crashes while writing to a card, it may automatically change the permissions to "Read Only." This can be very disconcerting, but all it takes is to umount and then run fsck on the card, and read permission will be restored once your Zaurus knows the file system has no errors in it.
But I do need to comment at this point that fsck is definitely not a harmless, reliable tool on the Zaurus. If the file system tables have been badly corrupted, it will be extremely wise to back up all the data on your card to another one before running letting fsck do any repairs. I have had instances where fsck has, instead of fixing the files correctly, totally corrupted more of them with each run. In most cases, I think this has occurred when my Zaurus has crashed while writing to the card, and that messes up the file system tables.
The best way to prevent this kind of disaster is to *AlWAYS* run fsck immediately after a crash which occurs when an expansion card is being written to, and also any time you remove a card from your Zaurus without first detaching the file system. And never, ever make any additions or modifications to the files or their names or add or remove any files from this card until the file system has been properly repaired.
Making any changes to the contents of an expansion card with compromised file system tables, whether it is ext2 or vfat, generally does more harm than good. I would run fsk on the card immediately, then remove the file that was being created or modified when the crash occurred (backing it up elsewhere if necessary), and then run fsck again, to make sure the card has a clean bill of health before making any further changes to it.
Sometimes, I have also found the problems with the file system on the card can be cleared up by rebooting my Zaurus, so if you have a card which does not seem to have a clean file system, you may want to see if the problem lies in the Zaurus rather than the card itself by rebooting before attempting to repair the file system with fsck.
If you find that you get "permission denied" errors when you try to perform write operations, on files that should be writable by you, examine the permissions. If write permission has been removed, it may look like your card is corrupt, but it may not be. I have this happen when I am working with a card that does have file system problems, properly remove that card, and insert a totally clean card. Then, at some point soon thereafter, I find that I cannot write to that clean card.
Examining the permissions, I see write permission has been removed, and when I try to restore it, the file system totally ignores my chmod command. I am learning that when this happens, I need to properly eject the card, and then reboot my Zaurus. After the reboot, when I reinsert the card, in most cases, write permission has automatically been restored!
Another thing that can cause file system problems is removing an expansion card physically without first unmounting the file system by either 1) using the GUI in your taskbar to eject the card, or by 2) using the "umount" command discussed above. If you accidentally remove a card, it will be very wise to run fsck the very next time you put the card back into your Zaurus, to make sure the file system tables are up to date, before you do anything else with the card.
Now, returning to explaining how to run fsck, here is sample fsck output from a brand new card, which came as vfat from the factory. First let us look at output from fsck.vfat with one possible set of options:
bash-2.05# fsck.vfat -f -a -t -v /dev/mmcda1
dosfsck 2.8 (28 Feb 2001)
dosfsck 2.8, 28 Feb 2001, FAT32, LFN
Boot sector contents:
System ID " "
Media byte 0xf8 (hard disk)
512 bytes per logical sector
16384 bytes per cluster
1 reserved sector
First FAT starts at byte 512 (sector 1)
2 FATs, 16 bit entries
125952 bytes per FAT (= 246 sectors)
Root directory starts at byte 252416 (sector 493)
512 root directory entries
Data area starts at byte 268800 (sector 525)
62856 data clusters (1029832704 bytes)
63 sectors/track, 32 heads
243 hidden sectors
2011917 sectors total
Checking for bad clusters.
Cluster 62857 is unreadable.
Reclaiming unconnected clusters.
Performing changes.
/dev/mmcda1: 0 files, 1/62856 clusters
bash-2.05#
Note that in this example, fsck finds 62856 data clusters, and complains that cluster number 62857 is unreadable.
Since it found 1029832704 bytes, and that is definitely more than a gigabyte, I will
not worry about it complaining that it cannot read more bytes.
Remember Sharp's fsck.vfat is broken, so it's output needs to be confirmed using other commands as well.
Then again, maybe the manufacturer just failed to mark the end of the partition!
What counts are the numbers and any other error messages that appear.
Remember that you will probably need root privileges to run fsck on a specific card. See the discussion above about sudo if you get an error like the following:
bash-2.05# fsck.vfat -v -t -a /dev/mmcda1
dosfsck 2.8 (28 Feb 2001)
dosfsck 2.8, 28 Feb 2001, FAT32, LFN
open /dev/mmcda1:Permission denied
bash-2.05#
Okay, now, on to a CF card example. Since my CF card is formatted with ext2, I will use fsck.ext2 to examine the integrity of my CF card. A clean card will show output similar to the following:
bash-2.05# fsck.ext2 /dev/hda1
e2fsck 1.19, 13-Jul-2000 for EXT2 FS 0.5b, 95/0
/dev/hda1: clean, 11/125488 files, 15851/500440
bash-2.05#
It is very, very easy to get misleading output from fsck and from fdisk if you do not use the correct syntax or version of the command. So, if your output looks different from the examples I have posted above or below, do not immediately conclude that the card is bad! If you get discouraging or alarming output, you may want to take a look at this page, where I have posted examples of misleading output due to benign problems and syntax errors.
If the card is a brand new one, or a used card that we want to reformat, we can use the "fdisk" command to show us whether it already has been partitioned, or has any bad spots on it (as hardware).
Please note that the "fdisk" command is a very powerful, but also dangerous tool, so you should not use it on any media that already has files on it unless you know what you are doing. One of my Linux mentors said that he always uninstalls "fdisk" after he finishes using it, just to make sure he does not do a lot of damage by accidentally invoking it when he does not intend to.
Just the same as with fsck, in order to run fdisk, you first must detach any possible file system by using the "umount" command. Remember to use "umount" without any "un". You then need to wait until the card icon in the bottom tray on your zaurus has disappeared, or no longer shows up in the output of the "df" command, before you enter the fdisk command.
So, for example, to run fdisk on an SD card, the commands you would enter to begin with would be:
bash-2.05# umount /mnt/card
bash-2.05# fdisk -l /dev/mmcda
Or, to run fdisk on an CF card, the commands you would enter to begin with would be:
bash-2.05# umount /mnt/cf
bash-2.05# fdisk -l /dev/hda
Now, since you just want to examine the card, I recommend you make sure to run fdisk with the "-l" flag, as shown above, so you do not take the risks associated with running it interactively.
On ROM 2.38, I have been unable to get an fdisk for any of my SD cards to produce a "perfect" example of what output from fdisk should look like.
So, here is what I believe the output for checking an SD card should look something
like (where I used "nnn" in lieu of the numbers that should appear):
bash-2.05# umount /mnt/card
bash-2.05# fdisk -l /dev/mmcda
Disk /dev/mmcda: nnn heads, nn sectors/track, nnnn cylinders
Units = cylinders of nnnn * 512 = nnnnnn bytes
Device Boot Start End Blocks Id System
/dev/mmcda1 * 1 nnn nnnnnn 6 FAT16
bash-2.05#
Here is a real example of the output when I ran fdisk on a new vfat SD card. Note that fdisk complained about a problem with Partition 1 on a brand new vfat card. This is just a warning that is not critical, which means there is a little bit of unused space at the end of the card. What counts are the numbers and any other messages that may appear.
bash-2.05# umount /mnt/card
bash-2.05# fdisk -l /dev/mmcda
Disk /dev/mmcda: 16 heads, 63 sectors, 970 cylinders
Units = cylinders of 1008 * 512 bytes
Device Boot Start End Blocks Id System
/dev/mmcda1 1 971 488840+
6 FAT16
Partition 1 does not end on cylinder boundary:
phys=(970, 2, 34) should be (970, 15, 63)
bash-2.05#
I do not recommend running fdisk interactively unless you definitely plan on reformatting the card and have read the resources I have listed below. However, in case you accidentally forget to put the "-l" flag in, you can enter the "p" command so that it will "print" (i.e. display) the card's contents. When it prompts you for the next command, you can enter a "q" to quit or exit without taking any action. Here is sample output from checking a CF card with fdisk interactively:
bash-2.05# umount /mnt/cf
bash-2.05# fdisk /dev/hda
Command (m for help): p
Disk /dev/hda: 16 heads, 63 sectors, 994 cylinders
Units = cylinders of 1008 * 512 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 993 500440+ 6 FAT16
Command (m for help): q
bash-2.05#
If you get discouraging or alarming output from fdisk, you may want to take a look at this page, where I have posted examples of misleading output due to benign problems and syntax errors for both fsck and fdisk. Most of the errors shown there have been due to using the wrong name for the media, so double check your "fdisk -l" to make sure that you use /dev/hda or /dev/mmcda without a number at the end of the device name.
If you are having serious problems with a card, the problem may be due to defects in the card itself. And the badblocks command was designed to examine media for such physical defects.
And, I have now learned the hard way that we should not assume that all computer media we purchase is free of defects at the point of purchase. Make sure you run the badblocks command on all new CF and SD cards and other memory devices before you start putting files on them. It will take time for low memory systems to scan the entire device, but will be well worth the time spent if your memory device is defective.
If your system has mounted the SD or CF card, then you must issue the "umount" card before running badblocks. The badblocks command does not need any special arguments, unless you want to send the output listing of badblocks to a file instead of to your console.
So, for example, to run badblocks on an SD card, the commands you would enter would be:
bash-2.05# umount /mnt/card
bash-2.05# badblocks /dev/mmcda
Or, to run badblocks on an CF card, the commands you would enter would be:
bash-2.05# umount /mnt/cf
bash-2.05# badblocks /dev/hda
If you want to run badblocks on media that has been mounted via USB on later Zaurus ROMs, the commands you would enter would be:
bash-2.05# umount /dev/sda
bash-2.05# badblocks /dev/sda
There will be no output from badblocks if no bad blocks are found, unless you specify verbose mode with the "-v" option. If there are many bad blocks, you may wish to route the output to a file, especially if you want to return it as a defective device and the vendor or manufacturer needs proof. But beware that with a severely defective card, the output can be quite voluminous. For example, the output from badblocks for a very defective 1 GB SD card I have, was 835K, and I had stopped the badblocks command as I did not want the full output!
If you wish to save the output from running badblocks on an SD card, and put the output in a file called "output.txt", the commands you would enter would be:
bash-2.05# umount /mnt/card
bash-2.05# badblocks -v -o output.txt /dev/mmcda
The badblocks command can also be used in combination with fsck and makefs to have them skip bad blocks, but I cannot comment on that feature as I have no experience with results from working with file systems that could be repaired in that way.
The idea seems to be that if you are having trouble with a card, you do everything you can to let fsck repair the problem. You can use the fdisk and the badblocks commands to help ascertain the nature of the problem.
On my Zaurus, I have had so much trouble with the file systems on expansion cards getting corrupted when the Zaurus crashed during a download, that I would strongly recommend backing up any crucial, difficult to replace files from the expansion card before you run fsck, unless you run fsck immediately after the crash or accidental removal of the card. Chances fsck will corrupt the contents of a card are slim if no other write operations have occurred since the crash, but otherwise, with both ext2 and vfat, the risks of doing damage can be quite high, with the versions of fsck we have for the stock Sharp ROMs on both the Collie and Tosa.
But remember that sometimes a file system that appears broken is not broken at all, and the Zaurus is just confused. So remove the card properly as discussed in the sections above, reboot your Zaurus, and reinsert and examine the card again. In many cases, the card will be fine after the Zaurus has been rebooted.
If you run fsck a few times with repairs enabled, and the problem is not resolved, then chances are that there is either a broken file system, or a hardware problem. In either case, the problem usually can only be resolved by backing up the contents of the card, repartitioning and reformatting it, and then copying your data back to the card. See the resources below for instructions on how to repartition and reformat a card.
Most of the above is true for the Raspberry Pi, as well. In the case of the Raspberry Pi, because there is no backup battery on the board, output indicating a broken file system can often be due to an indequate or erratic power supply. Before you give up and assume that the card is bad or it's file system has been corrupted, sync and reboot your Pi, and test your power supply. If you can do so, it also may be wise to check the media on another computer, to see if you get the same results from fsck and badblocks.
If you find any errors or unclear sections on this page, please notify me.
Here are some resources with more information related to this topic:
http://www.oesf.org/index.php?title=SD_And_CF_FAQ (see note below)
http://www.oesf.org/index.php?title=Step-by-step_CF/SD_fdisk/formatting_for_newbies (see note below)
http://tldp.org/LDP/abs/html/system.html
Partitioning with fdisk
http://tldp.org/HOWTO/Partition/fdisk_partitioning.html#fdisk
Recovering a Deleted Partition Table
http://tldp.org/HOWTO/Partition/recovering.html
NOTE: The wiki pages at oesf have been subject to spammers. If you find either of the above two pages at oesf to have been defaced, please notify an admin at oesf or log in and restore the page if you are registered at the wiki. If you urgently need the information, you also should be able to retrieve it from the Wayback Machine at http://www.archive.org/index.php, or you can contact me.