CF / SD and large drives FAQ

fgh

Member
AmiBayer
Joined
Nov 28, 2010
Posts
707
Country
Norway
Region
Lindesnes
.
Introduction
This is not meant to be a step-by-step guide to set up a large SD or CF card as an amiga system drive, but it will help you identify many of the common issues.
Parts of this are based on info collected from other EAB threads, often with research by Toni Wilen, Thomas Rapp, or others.
I've recently moved the 'max drive capacity' to the top. You'll find everything else is still there, just a bit further down...



Maximum drive capacity

The 4GiB limit
Description: Max HDD or card capacity 4GiB (total drive size).
Applies to: Any IDE or SCSI drive or CF/SD card used without a 64 bit file system
Reason: Maximum with 32bit addressing
Sources: Commodore's FFS file system, plus some parts of Workbench.
Consequence: Writes above the 4GiB threshold will 'wrap around' and destroy data and the RDB on your device.

The 7,87GiB limit
Description: Max HDD or card capacity 7,87GiB (total drive size).
Applies to: A600/1200/4000 IDE with modern filesystem but original IDE driver (scsi.device)
Reason: CHS addressing is artificially limited to 7,87GiB (16383/16/63) by the ATA-specification, therefore larger drives still report 7.87GiB size when accessed with CHS. (CHS was made obsolete in ATA-6 in 2002)
Source: Scsi.device, as it only supports CHS addressing. (But when given larger than ATA-standard CHS values, it works correctly up to 128GiB.)

The 128GiB limit
Description: Max HDD capacity 128GiB (total drive size).
Applies to: A600/1200/4000 IDE with modern filesystem and many 'modern' versions of scsi.device (os 3.5/3.9 versions, etc..)
Reason: Maximum with 28bit LBA addressing. 48bit LBA (ATA-6, 2002) is required for drives larger than 128GiB.
Source: Scsi.device, as only recent versions of scsi.device support 48bit LBA (v43.45 and v44.2, see table below)​


Solutions/Recommendations
4GB cards: You can use original scsi.device and FFS if you want, but I recommend using a modern file system (SFS/PFS3), as it's faster, safer and more elegant (no endless validation, etc).

8GB cards: These are smaller than 7,87GiB, so the easiest is to use original scsi.device with a modern file system supporting direct-SCSI mode. (I recommend PFS3 All-in-One, but PFS3 (DS) or SFS v1.84 also work). If you choose to patch scsi.device, file systems using TD64 or NSD also work, but check the table below to make sure the file system is compatible with the scsi.device version. (PFS3 AIO is compatible with all)
Note: It is possible that '8GB' cards exist that are larger than 7,87GiB (= 8.45 GB = 8063 MB = 8455200768 bytes). I've tested ~15 so far, and they were a lot smaller though. If you have such a card, please let me know, and restrict the drive geometry below 8455200768 bytes or use a scsi.device patch)

Above 8GiB: You generally need to both use a modern file system and patch the IDE driver, scsi.device to a newer one supporting LBA addressing.
(There is one exception: the IDE-SD adapter described below.)

From the table below you will see that the only versions of scsi.device to support 48bit LBA is v43.45 and 44.2.
As some people claim they have issues with 44.2, I recommend version 43.45 for now. (Although v44.2 is faster, and worked 100% for me)
You load the updated version using 'LoadModule' in your startup-sequence.

IDEfix supports drives up to 128GiB, but doesn't like removable type CF cards, as described under 'Removable vs fixed flash media', below.​


Overview, IDE drivers and file systems
attachment.php


This table displays some limitations and compatibilities between amiga IDE drivers and file systems.
As you will see, not all combinations will work. I recommend PFS3 AiO edition, because PFS3 is great, and the AIO version works with all CPU types, scsi.device versions and has a workaround for the maxtransfer issue.​


Direct SCSI, NSD and TD64?
All amiga .device drivers use the commands CMD_READ and CMD_WRITE that are limited to 32bit addresses and therefore a 4GiB drive size.
The command HD_SCSICMD (aka Direct SCSI) uses block addresses, and can therefore address 2TB (512 times more).
At one point TD64 (Trackdisk64) was created, with TD_READ64 and TD_WRITE64 commands, supporting 64bit addresses and therefore (theoretically) 16 exabytes.
Later the functionally identical but incompatible NSD was introduced, and NSCMD_TD_READ64 and NSCMD_TD_WRITE64 was supported in AmigaOS 3.5/3.9.
To use either NSD and TD64 requires both a device driver and a file system that supports it. (See the table above.)

Then there is NSDpatch that can patch NSD support in to various old drivers, but:
Note that scsi.device from 3.1 and before is limited to 7,87GiB due to CHS addressing, even if you use direct SCSI or NSDpatch.
And newer versions of scsi.device not supporting 48bit LBA are also limited to 128GiB (28bit LBA).​


Other problems > 4GiB
As mentioned above, some parts of Workbench also have issues with devices above 4GiB..

HDToolbox only ever writes to the RDB of your drive, and it accesses your drive through the device driver (scsi.device).
With correct drive geometry, all versions of HDToolbox can set up drives up to 2TB (two terabyte).
The 'read configuration' function will be limited by the device driver (scsi.device) though.
This means that geometry will only be read correctly on devices up to 7,87GiB without patching scsi.device. (8GB cards are slightly smaller than this)
(With 'illegal' CHS values as with the SD-IDE adapters mentioned below, it detects up to 128GiB correctly)
Oterwise, if you write in your own values, or the drive has an RDB with correct geometry, (set up in WinUAE or otherwise), any version of HDToolbox is able to partition cards above 7,87GiB without patching scsi.device first. (But you still have to patch it later, to be able to access those partitions)

HDToolbox from older than OS3.5 also has some none-critical issues you need to be aware of:
It will report the total drive size incorrectly for drives above 2GiB. (showing some negative value).
When using 'read configuration' with a removable type device, it will say "Error:Unit is not a disk (Type 7)".
You do need to do some math when creating large partitons, as the partition size counter resets at 4096MB.
These issues are simply annoyances, and can be ignored.

The AmigaOS format function accesses the device directly, and is limited to 32bit.
Therefore it will also 'wrap around' and corrupt data and partitions when formatting a partition above the 4GiB limit. This is fixed in OS3.5
So always use quick format on any partition above the first 4GiB of any drive. (Actually, always use quick format anyway..)

Other programs accessing the harddrive directly will also destroy your data and partitions when accessing >4GiB, even if a new file system and scsi.device is setup correctly. (ReOrg, DiskSalv, AmibackTools, QuarterbackTools, DynamiCache, ShapeShifter, PC-Task, PCx and others) So make sure these operate on partitions below 4GiB..
This doesn't apply to Elbox drivers (for 4xEIDE'99 and FastATA adapters) if used in SPLIT mode, as this mode accesses large drives as multiples of 4GiB, so normal format and such programs are again safe to use)​


Size of boot partition
A bootable partition can be of any size as long as the entire partition is accessible before any scsi.device patches.
(= First 4 GiB w/o scsi.device patch or directSCSI filesystem, first 7,87GiB with dSCSI, 128GiB with dSCSI and clever IDE-SD, or entire drive with a custom burned 3.9 ROM)

But if you have more than 2 or 4GiB free space on a partition, lots of old installation programs will fail when calculating free partition space, stating 'not enough free space' or similar. (32bit overflow again)
There are workarounds such as installing to RAM: and moving the installation afterwards, or even filling up the partition with stuff before and remove after (!), but this quickly gets tedious.
So I suggest you keep system partition size less than 2GiB. (Not nessecarily the first 2GiB, as long as the end is accessible before scsi.device patch)​


OS 3.9 Boing Bag 2 issue
Scsi.device v43.43 from OS3.9 Boing Bag 2 attempts to support 48bit LBA addressing required for drives larger than 127GiB.
Unfortunately it was not implemented correctly in v43.43, and only later fixed by Toni Wilen in v43.45.
OS3.9 BB2 uses its buggy LBA48 code with any drive above 4GiB that reports LBA48 capability, and will guru immediately after BB2 setpatch has rebooted the system.

If you have such a drive/card and want to use OS3.9 BB2, the solution is to skip the scsi.device rom update from BB2. There are at least two ways to achieve this:
1 Loading a newer version of scsi.device (43.45) with LoadModule >NIL: DEVS:scsi.device NOREBOOT and Setpatch skipromupdates "scsi.device" QUIET
2 Using maprom feature or an actual EPROM to load a custom 3.9 ROM with a newer scsi.device and SetPatch NOROMUPDATE QUIET​


Clever SD-IDE adapters
There is a common and cheap SD-IDE adapter that looks like this, that has a nice feature:
It lets you use 16 or 32GB SD cards without patching scsi.device.
This works because the adapter provides 'illegal' CHS values (larger than ATA-maximum 7,87GiB), communicating instead the correct total drive size.
The adapter reports as 'removable' though, so IDEFix and other drivers by Elaborate Byte will fail with it.

There is a newer version of the same adapter, with SDXC support (cards larger than 32GB). It looks like this.
This one reports as a fixed device, so it works with IDEfix etc, but it does not give CHS values above 7,87GiB anymore.​


The CHS drive size issue
Problem: If, on a system limited to CHS addressing, you access a drive that was initialized and partitioned on an LBA capable system, you will get in trouble if the last partition of the device is extended to the very end of the drive.

Applies to: Devices of any size up to 7,87GiB (including 8GB CF/SD cards), initialized in any way in WinUAE, or in OS3.5/3.9 HDToolbox on real HW, and then used somewhere else without a scsi.device patch.
Also applies to 16/32GB SD cards used without scsi.device patch on clever ide-sd adapters, and to IDE devices initialized on a SCSI-IDE adapter with OS3.5/9, and moved to IDE without scsi.device patch.

Reason: Total drive size by LBA is exact, but by CHS it is limited to multiples of Heads and Sectors-per-track.
It is common for devices to report the full size with LBA, and a slightly smaller size with CHS. Hence, with CHS you are not able to address the last blocks of the drive.
WinUAE does not retrieve the size information directly from your device, but uses what's provided by Windows, so even when using WinUAE's IDE HD-controller you get the full size.

Consequence: With FFS or regular PFS you will get read/write errors when you attempt to access the last area. (IE when the last partition is ~half full, as partitions fill up from the middle)
With SFS or PFS AiO the last partition will not mount.
(It will not destroy data, you'll simply be unable to read from or write to this area.)

Solution: Patch scsi.device, or reduce the RDB size definition by 2 MiB or more, or leave 2 MiB or more unused at the end of the drive.
Or you can initialize the drive on a real amiga with WB3.1, or with OS3.5/3.9 using HDToolbox from WB3.1 or older.
It's just the drive definitions that need to be done in the old version, the partitioning can be done in WinUAE or OS3.5/3.9 HDToolbox.
(This 'old hdtoolbox in OS3.5/3.9' scenario is not thorougly tested, but as long as HDToolbox reports maximum 16 Heads and 63 Bytes per track you're fine)

Why 2MiB? 2MiB is largest discrepancy between CHS and LBA size. For ATA compliant devices it's only 504KB (16 H x 63 SPT x 512 B), but up to 2MiB within CHS theoretically (with 256 Heads).

If you want to check if your existing drive is affected, run a program that can read the RDB and check if the stored CHS values have more heads than 16 or more Blocks/Sectors per track than 63.

Update: A related issue submitted by Thomas is that some SD/CF cards with very poor firmware forget that the first LBA block is number 0, and consequently tries to access one block too far.
This would also fail with the same consequences when used in LBA capable system​


Capacity limits other than a600/1200/4000 IDE
  • No amiga setup can safely access > 4GiB without a 64bit file system.
  • This FAQ details a600/1200/4000 IDE, but some third-party IDE drivers and older SCSI controllers have other, lower limits.
  • Using PCMCIA with FAT95, PFS3 or SFS, there are no capacity limits, as you're using compactflash.device and TD64 compatible file systems.



General information

WinUAE considerations
If you want to access a physical amiga drive/card in WinUAE, you need to run the program as administrator (except in WinXP).
Second, Windows does not allow applications (WinUAE) to overwrite the MBR of active windows partitions.
So in order to write amiga partitions to the card, you need to remove all windows (MBR) partitions from the card first.
  • In Windows Vista/7/8, the diskpart commandline utility supports removable media (use 'clean' command).
  • For Win XP you need a tool like Partition Wizard or EASUS Partition Master.
  • On Linux you can use the dd command
  • On Mac OS use disk utility (Partition -> 1 Mac OS partition -> Free space)
Finally, when prepping a drive smaller than 8GiB in WinUAE, leave at least 2MiB of unused space after the last partition, as the last two megabytes of the drive might not be accessible on real HW. (More info under 'CHS drive size issue', above)

Some WinUAE options explained
For accessing amiga flash media in WinUAE you can select either IDE or UAE hd-controller.
If you select IDE, WinUAE will use scsi.device IDE driver from the kickstart file.
This way you won't need to change any HDToolbox tooltypes, and you might spot some errors already in WinUAE.
If you are done prepping the card and want faster file transfers, you can use 'UAE'.
(Using 'UAE', HDToolbox can still access the card but you need to change hdtoolbox's SCSI_DEVICE_NAME tooltype to uaehf.device.)

Later versions (>3.0.0) also have CF/HD choice for hardfiles. A 'CF' type hardfile will indentify itself as typical removable CF card, and will trigger associated errors (idefix failure, 'Unit is not a disk (Type 7)' in hdtoolbox, etc)
Leave it at 'HD' unless you have a specific reason not to.

Hardfiles also have an new ATA level option, deciding to which standard the drive will adhere. ATA2+ is default and will give you LBA48 support (>128GiB) and no max transfer errors.
The other options are ATA1 (no LBA48, no max transfer problem) and ATA2+ Strict (LBA48 support, max transfer problem will occur)
As with the CF/HD option, these features can trigger errors that weren't reproducable in WinUAE before, and can be useful, but I suggest you use them with caution.​


Max transfer and mask
  • First: The max transfer setting limits the max size of one file transfer. It has no effect on speed.
  • When using onboard IDE and scsi.device, a maxtransfer setting of 0x1fe00 (or lower) is required on all partitions with any file system.
    Be aware that wrong (too high) values are mentioned lots of times on EAB!
  • The reason it must be limited is that higher values will lead to read- and write errors, and data will be corrupted. (Changing to the correct value will not get corrupted data back..)
  • This is due to scsi.device's outdated ATA1 behavior. (incompatible with newer ATA standards). The problem is described here.
  • Specific max transfer settings are not required if you use IDEfix97 on internal IDE, when using PFS3 All-in-One or when using PCMCIA
  • I do not know which drivers for third party IDE controllers are affected. It is likely that drivers made before 1994 (before EIDE, and later ATA-2 in 1996) are affected.
  • Mask settings are for DMA capable controllers and not required for Amiga IDE or PCMCIA use.

Removable vs fixed flash media (short version)
Original scsi.device IDE drivers works well with both fixed and removable type CF cards.
The Amiga PCMCIA driver compactflash.device also works with both types of cards.
Some CF cards do not work with compactflash.device, but this is for other reasons.

IDE drivers confirmed NOT working with removable type CF cards:
  • IDEfix (Elaborate Bytes)
  • Buddha (Elaborate Bytes/Individual Computers)
  • XSurf (Elaborate Bytes/Individual Computers)
  • Catweasel (Elaborate Bytes/Individual Computers)
When loading these drivers (usually during boot) with a removable type card, the amiga will hang/freeze.

Doobrey has made patches for IDEfix and the other Elaborate Bytes IDE drivers, that correctly identify CF cards, and allow removable type cards to work correctly.

SD cards are neither removable or fixed in themselves, as they are not ATA devices.
It's therefore up to the IDE-SD adapter which response is given. (Examples under 'Clever SD-IDE adapters' above.)

See 'reasons for incompatibility' below for more information..​


HDinstTool vs HDToolbox
Never use HDinstTool with a drive that was partitioned using HDToolbox if you want to keep any data on it.
HDinstTool has a nasty bug that will destroy partitions that are not defined and saved by HDinstTool originally.​


About wear levelling
Due to the concept of 'wear levelling' on flash media, it is smart to avoid block-by-block writing, like a normal format, or writing back an image file over the entire card.
On a hard drive, if you write to a certain block, this exact physical block will be written to every time.
On flash media, the firmware will spread out the writes (and keep track of them) to even the strain on the fragile flash cells.
When a cell is damaged (worn out or faulty), the firmware will notice, and move the data to an unused cell, and avoid using the damaged cell again.

The problem with a full format or writing an image over the entire card is that it writes to ALL the blocks and makes the card's firmware consider them all used (It doesn't know your file system).
Therefore none of those cells can be used as replacement for damaged cells any more, and data will be corrupted faster.
Most cards have some over provisioning, meaning some extra cells available to the firmware for such tasks. But you never know if older/cheaper cards do.
So always use quick format, and if you'd like to maximise the lifespan of your card, copy files to it rather than writing an image to it.
(And don't even think about low level format, there is -physically- no way of low level formatting flash media.)
You could also leave some space unpartioned at the end of the drive, to avoid using all the cells. (= Your own over provisioning)

You might have heard about TRIM. This is used to keep the firmware informed when blocks become unused again, to optimise wear levelling to improve life span and performance.
But TRIM is only available on SSDs and requires software support that does not exist for Amiga.​



Removable media troubleshooting

My CF or SD works in WinUAE but not in my real Amiga
This is often caused by a WinUAE configuration that does not match the real hardware. (amiga model, kickstart versions, ram, etc), so certain software would work in WinUAE but not on real hw.
If the filesystem that you have installed requires a certain CPU or kickstart that your real hardware does not have, the machine would guru before it starts booting the hdd.
(SFS requires 68020 and KS2.04, PFS3 works with 68000 but requires KS2.04, PFS3AiO works on 68000 and KS1.3)

Another filesystem scenario is when using a drive larger than 4GB with original PFS3 version in non-DS mode.
Unless you are using IDEfix, original PFS3 needs to be in DS mode. (See compatibility table above, PFS3 supports TD64, not NSD)
The reason it would work in WinUAE is that WinUAE gets the IDE drive size from windows, and is not limited by lack of correct TD64/NSD support. (Applies to both UAE and IDE hdd-controller selections).
Solution: Use PFS3DS or PFS3 AiO version by Toni Wilen instead.

Otherwise you might have other software installed that is incompatible with your real amiga hardware. In this case you should still have access to the drive if booting without startup-sequence:
Hold both mouse buttons down during boot, check that all expected partitions show up, select boot without startup-sequence and check that you have access to the boot partition.
(Access to partitions above 4GB will not be available until a scsi.device patch has been run)
Then you can troubleshoot your startup-sequence etc to figure out what's causing the problem.

Another scenario that came up a couple of times after I started this FAQ is that the area that WinUAE accesses is not starting entirely at the beginning of the drive.
A couple of guys ended up with a drive without windows partitons, where WinUAE still didn´t have access from the very beginning. In both cases there was 8MB of unused space before the first partition, indicating that the drive might have been GPT formatted, and not entirely recovered.

If you don't think these are the reasons:
Try hdtoolbox on the real hardware (boot with workbench install disk or any other wb installation).
Make sure that hdtoolbox is using the original scsi.device (SCSI_DEVICE_NAME tooltype) and that you are not running unpatched IDEfix, or other IDE drivers.

If the drive shows up, you might have to check your favourite hd prepping tutorial. If your drive is 4GB or smaller, or you know the geometry so you can enter the size values manually, you could quick format it on the amiga before filling it up in WinUAE.

If it does not show up in hdtoolbox, or does not work reliably, It might be one of those cards/drives that are simply not compatible with the old drivers, and I have no suggestions at this point. (Suggestions please!)


Hdtoolbox says 'Unit is not a disk (Type 7)'
Don't worry, nothing is wrong. Your removable type card is mistaken for an optical removable media (CDR, etc) by hdtoolbox.
This occurs with all cards that report as removable media (848A), when you go to 'change drive type' and select 'read configuration'.
Just ignore the message, fill out your own manufacturer and model info and proceed to partition the card.
(HDToolbox from wb 2.x/3.0/3.1 will show negative capacity for large drives (>2gb?). Ignore it, don't worry unless the partition page shows the wrong size.)
Also, if you use WinUAE to prepare the cards you should not have this issue (even if you mount the CF using IDE0).


Rare issue: Drive shows up as 'no disk in drive' after partitioning?
You have partitioned the drive normally, then after a reboot, instead of showing up as NDOS / not a dos disk, some or all FFS partitions don't appear on workbench at all, and C:Info lists all missing partitions with "no disk in drive" state. (HDToolbox still shows all partitions correctly.)

This is a rare DOS/FFS bug described by Toni Wilen.
If the first 4 bytes of the first block of the partition contains bytes $FFFFFFFF, the FFS file system gets confused and thinks there is no disk.. (It might confuse dos disk type $FFFFFFFF = -1 = ID_NO_DISK_PRESENT with real dos type)
This bug can trigger even if drive is new, for example some DOM's or solid state memory cards may contain random looking non-zero data from factory. (Test patterns?) Fix is to wipe the drive completely, or better: Use this little program by Thomas.
OS3.9BB2 works (reports correct "not a dos disk" state), I haven't tested OS3.5-OS3.9BB1. 3.1 and probably older versions have the bug.​


Booting with Boing Bag 2 freezes system
If your OS3.9 Boing Bag 2 system manages to get to the setpatch reboot, but freeze immediately after the reboot, you may have fallen victim to scsi.device 43.43's LBA48 bug.
See 'OS 3.9 Boing Bag 2 issue' paragraph above.​


CF or SD does not work in PCMCIA adapter
First - you need to install compactflash.device (pcmcia CF driver) and Fat95 (file system).
For automount, move CF0 from Storage/Dosdrivers to Devs/Dosdrivers.
And make sure to use the latest version as compatibility increased over the years.
Specifically, if your machine freezes when inserting the card/adapter, the first thing you should do is update to the latest version of compactflash.device. (v1.32 at time of writing)

Your amiga can access cards in FAT (windows) format or in amiga formats through the PCMCIA.
It is most common to use FAT, so your PC can access the card as well.
The CF0 mountfile specifies the file system. If CF0 is set up with Fat95 (default), you can read a FAT16/32 card.
If you want to read a card with amiga filesystem, you need to change filesystem in the CF0 mountfile PLUS define the size etc of the card.
(FAT95 does this automagically.) You may use Giggledisk on aminet to create the mountfiles.

FAT-formatted card (accessible on pc) shows up as CF0:NDOS in PCMCIA adapter
New cards come formatted either as superfloppy (without MBR partition table) or with a MBR partition table with one partition.
It seems that Fat95 accepts both types but not all variants.
Specifically FAT12 format seems to show up as NDOS, but there may be others as well. Let me know if you find out :)
Anyway, quick-formatting the card on the amiga should solve it.

Other possible problems

If your amiga has kickstart 3.1 and a memory expansion or accelerator that can take maximum 8MB RAM, adding any fast RAM above 4MB will conflict with PCMCIA address space and the PCMCIA port will be disabled.
Many cards of this type have a jumper to limit RAM to 4MB to avoid this conflict. Use it.
Kickstart 3.0 does not disable the PCMCIA port, so storage and I/O cards will still work with 8MB, but any PCMCIA SRAM will conflict with other fast RAM >4MB.

A1200 also has an PCMCIA issue that requires reinserting the card after a reboot. (Hardware (Gayle) changed, but kickstart ROM did not..)
CFD v1.21 (2003) and newer fixes this. (As does the cardreset software). A600 does not have this issue.

People often suggest installing Cardpatch when troubleshooting CF over PCMCIA.
I don’t know if it is required for storage media, but it will not harm. (Applies to both a600 and a1200)


SDHC card does not work in PCMCIA SD- or multi-adapter
So far I have not found a single PCMCIA adapter that supports SDHC cards (4-32GB).
All the SDHC compatible adapters I have found are 32bit Cardbus, not 16bit PCMCIA.
Look specifically for a 16bit PCMCIA adapter supporting SDHC (SD High Capacity), and let me know when you find one!
For SD cards (2GB and below) there are still cheap PCMCIA adapters around, but expect them to become increasingly rare.​


Only my first FAT partition partition shows up
FAT95 does not automatically mount multiple partitions.
You need to create seperate mountfiles for these. (See giggledisk on aminet)​



Reasons for incompatibility

Background
The ATA standard supports lots of different types of media, both fixed and removable types.
IDE drivers communicate with devices using the ATA protocol, and require devices to identify as removable media or not.
For the CF card itself this type info does not change anything, but it has to give an answer.
Many CF cards are set to report as removable media, but not all, and it's hard to know in advance. ('Industrial' level cards are usually fixed type)
The information is stored on CF card's CIS (Central information structure) and communicated by the ATA controller inside the card when replying to the ATA IDENTIFY DEVICE command.​


The problem with certain amiga IDE drivers and 'removable' media
As mentioned above, original scsi.device IDE drivers have no problem with removable type CF cards or SD cards.
However, drivers by Elaborate Bytes - idefix97 and individual computers product drivers - do not work with removable type devices.

The drivers do not comply with the ATA-4 (or newer) specification.
IDE drivers use the ATA command IDENTIFY DEVICE to get all kinds of information from the device.

The first word of the device reply (word 0, 'General configuration') is 848Ah for removable type CF cards, and 044Ah or 0040h for fixed type CF cards.

attachment.php


First, bit 15 set in the reply (cards reporting 848A) can cause drivers to fail.
Drivers not written in accordance to ATA-4 (or newer) do not expect bit 15 set for ATA devices.
  • ATA-4 standard ( 1998 ) says that bit 15 in word 0 shall be 0 for ATA devices (page 95), but can be 1 (848Ah) for CF compatible cards (page 52).
    It also says that ATAPI devices shall not respond to IDENTIFY DEVICE, but reply to IDENTIFY PACKET DEVICE instead (page 41)
  • ATA-3 standard ( 1997 ) says it must be 0 for ATA devices and 1 for ATAPI devices (page 49)
  • ATA-2 standard ( 1996 ) says it is 'reserved for non-magnetic devices' (page 37)
  • ATA-1 standard ( 1994 ) says it is 'reserved for non-magnetic devices' (page 44)
  • CF specification v3.0 ( 2004 ) says word 0 standard is 848Ah, but can alternatively be 044Ah or 0040h to "turn off removable media". ( page 130 )
Second, bit 7 set indicates that a device is removable media. This did not change since the original ATA-1 specification.
(Additionally bit 6 could be set to specifically mark a disk as fixed, but this was removed in ATA-6 standard (2002) so I guess modern drivers use bit 7.)

See attachments
Attached are ATA specifications 1-5, CF spec 3, and the entire IDENTIFY DEVICE replies from my four CF cards and one SD card/adapter.

Amiga IDE drivers that fail with removable type CF cards probably expect bit 15 to be 0 for ATA devices, and therefore either give up on the device, or regard it as an ATAPI device and fail.
It's confirmed by Doobrey that Elaborate Bytes drivers simply stop processing the device if bit 15 is set in the reply.
It's possible that other drivers might accept bit 15 set but still fail with removable type CF cards because bit 7 is set.

A CF card's reply can not be changed manually except on a few old cards where the manufacturer (Sandisk, Lexar) provided tools to ‘flip the removable bit’.
My Transcend 4GB 133x, Kingston 4GB (flower img) and Kingston 16GB 266x cards report as fixed (word 0 = 044A)

The ATA Identify Device replies can be read with the BusTrace program.

J.Schönfeld made the TrueIDE dual CF-IDE adapter that works around the problem with his amiga drivers. (And also lets windows users partition and boot from removable type CF cards)
This adapter modifies the CF cards IDENTIFY DEVICE reply to allow removable type cards on drivers that only work with fixed CF cards.​


Regarding PCMCIA operation and incompatibility
Compact Flash storage devices operate in three modes:
  • PC Card Memory Mode
  • PC Card I/O Mode
  • True IDE Mode
Compact Flash devices must support operation in all three modes, but operate only in a single mode at any given time.
The CF card senses (using pin #9) if it's connected to IDE or PCMCIA, and switches modes accordingly.
Connected to IDE, it uses True IDE mode, and becomes a hardware-level PATA IDE device.
Connected to PCMCIA, things are getting more complex, as it takes the role of both IDE controller and IDE device, using PC Card I/O mode.
CF cards needs to support several different PCMCIA access modes, different register locations, etc.

Some CF cards (Kingston 4Gb etc) work with IDE/scsi.device but not with pcmcia/compactflash.device.
This can theoretically be due to the card, the driver or the amiga pcmcia implementation.
It is confirmed (by Toni Wilen) that some cards do not support all mandatory register locations, causing problems with amiga pcmcia operation.

Let me know if you have more information :)


Regarding SD cards
SD cards are not ATA devices and have less complex controllers than CF cards.
Therefore there are far less differences between SD cards than between CF cards.
For the same reason SD-IDE adapters need to be active adapters, not just passive like most CF-IDE or CF-PCMCIA adapters.

As I wanted to experiment with SD cards, I have successfully tested 12 different 8GB SDHC cards on my a600 (PFS3 direct-scsi and ROM resident scsi.device) with a £6 SD-IDE adapter.

When connected to a PC (over IDE), that adapter report as a removable type CF card (word 0 = 848Ah).
Therefore, drivers having problems with removable type CF cards will also have problems with such adapters.

I have later tested the newer SDXC capable version of this SD adapter, and this one reports as a fixed device.​


Relevance of PCMCIA / CF card voltage ratings?
The PCMCIA specification allows 3,3 or 5 volt devices, and the amiga pcmcia implementation only works with 5v devices.
The ‘low voltage key’ (physical incompatibility) on 3.3v pcmcia devices prevent using a 3.3v device in an amiga.
But if a 3.3v only CF card existed, and was used in a generic PCMCIA adapter, this could be a problem…
However: According to this document on the Compact Flash Association, all CF cards are able to operate on both 3.3v and 5v operation.
So voltage rating should not be relevant.​


That’s it for now, well done if you are still reading!
There certainly are incompatible cards, but I hope some will find their card is compatible after all with this little guide.
Additions and suggestions are welcome! Thanks :)
 

Attachments

  • ANSI ATA-1 1994.pdf
    134.3 KB · Views: 5
  • ANSI ATA-2 1996.pdf
    399.2 KB · Views: 2
  • ANSI ATA-3 1997.pdf
    778.6 KB · Views: 1
  • ANSI ATA-4 1998.pdf
    1.4 MB · Views: 1
  • CF spec 3 2004.pdf
    1.4 MB · Views: 2
  • id_pny32mb.txt
    6.6 KB · Views: 3
  • id_transcend4gb.txt
    6.7 KB · Views: 0
  • id_kingston4gb.txt
    6.7 KB · Views: 0
  • id_transcend8gb SD.txt
    6.7 KB · Views: 0
Last edited:
An excellent guide and as such I've taken the liberty of making it a sticky (y)

Dave G :cool:
 
This is a fantastic write up and I shall sticky this NOW

d'amn... beaten by the Davideo! ;) (y)
 
Another common problem is file corruption when copying to/from CF cards in PCMCIA.
I get that alot, mostly on the faster machines (040/060).
I guess that is probably due to the driver, not the card.
 
Original post updated with some more info:
Removable bit is not relevant with original scsi.device (onboard IDE) and compactflash.device (PCMCIA).
However IDEfix97 drivers and drivers for IDE controllers from Indidual computers require fixed cards (unless patched).

Another common problem is file corruption when copying to/from CF cards in PCMCIA.
I get that alot, mostly on the faster machines (040/060).
I guess that is probably due to the driver, not the card.

Ok that's new to me.
I've heard about file corruption on scsi.device in relation to drives above 4gb without proper software / patches, or when maxtransfer is not set correctly.
But neither should apply when using PCMCIA.
  • Could it be the other drive you are copying to/from?
  • Are you using compactflash.device, fat95 and a FAT16/32 formatted card, or something else?
  • Does the file corruption occur both when copying from PCMCIA to IDE, from IDE to PCMCIA and copying within same card on PCMCIA?
  • You say it is a common problem. Do you have a link to some other discussions where this problem is described?
 
Last edited:
Found something else this evening that may be of help to ppl with dodgy CF's.

Was constantly having errors on cheap CF's when writing to the 2nd partition.

Thought I'd try the CF as just one big partition, no errors, card is now full o games and running sweet, no transfer rate changes etc.

Spooky, perhaps some cheaper CF's just don't like multiple partitions?
 
Was constantly having errors on cheap CF's when writing to the 2nd partition.

Thought I'd try the CF as just one big partition, no errors, card is now full o games and running sweet, no transfer rate changes etc.

Spooky, perhaps some cheaper CF's just don't like multiple partitions?

Hi there,

  • Are you using PCMCIA and compactflash.device, onboard IDE and scsi.device, or something else to access these?
  • What size are the cards and partitions?
  • Did you set the maxtransfer on the second partition to 0x1fe00 (and pressed enter afterwards :))?

I didn't think a CF in itself could support multiple partitions or not. An MBR can limit partitions though. A removable type device can not (easily) be given a MBR supporting more than one partition on Windows, but this is not based on a real difference in the card, just the removable bit from the CF controller's reply.

Scsi.device and compactflash.device disregard the removable bit.
An amiga filesystem disregards any MBR, but fat95 needs one, so if you use PCMCIA and fat95, it is possible that the MBR might be causing this.

If this, maxtransfer or drive/partition above 4gb is not the cause, I don't know what might cause it..
 
Just using it via IDE. Don't have pcmcia port to play with :)
 
ok.

What size are the cards and partitions?
Did you set the maxtransfer on the second partition to 0x1fe00 (and pressed enter afterwards )?
 
Tried this method on a 1gb and a 2gb card. Max transfer left at the default hdtoolbox comes up with.

My 4gb sandisk works happier this way also. Really weird stuff, would be great if anyone with a dodgy card could test this out. :)

When trying 2 partitions I use the value above in your faq. Odd stuff.
 
Ok, just to be clear: Maxtransfer setting of 0x1fe00 (or below) is required for all partitions, no matter how many partitions you have. (When using onboard IDE & scsi.device)
When entering the maxtransfer setting, you also need to press enter while the cursor is in the input field, otherwise it will not save the value. (Strange old fashioned GUI behavior)

If you use a higher value you will have file corruption when writing files larger than 128kb. (As long as the copy program has a buffer size >128KB and file system is not too fragmented)

If you are sure this is done correctly, I agree it is weird. :)
 
Ok, just to be clear: Maxtransfer setting of 0x1fe00 (or below) is required for all partitions, no matter how many partitions you have. (When using onboard IDE & scsi.device)
When entering the maxtransfer setting, you also need to press enter while the cursor is in the input field, otherwise it will not save the value. (Strange old fashioned GUI behavior)

If you use a higher value you will have file corruption when writing files larger than 128kb. (As long as the copy program has a buffer size >128KB and file system is not too fragmented)

If you are sure this is done correctly, I agree it is weird. :)

Yeah, for all partitions I usually use 0x1fe00 on the decent cards that haven't failed me before.

Just these generic cheap cards that seem to get on better with the standard values and using 1 partition.
 
Ok, that does not make any sense to me..
Edit: No card should work better with maxtransfer higher than 0x1fe00. This only has to do with scsi.device, not the card.

Would be great if anyone with a dodgy card could test this out. :)

I' be very happy to test out those failing cards if you would send them here. I'd pay shipping and a few euros for them :)
 
I can post one of them off, the other is in use :)
No charge mate, as I'm curious to this also.

Only possibility would be, does it make any difference that it's being used on a AlfaPower 500 and not the standard IDE?
 
It might make a difference, I didn't notice you were on an a500. (Should have read your signature..)
As I wrote in the first post I don't know status of maxtransfer problems in third party IDE controller drivers.
But chances are old ones like yours also expect ATA-1 compliant devices and therefore require 0x1fe00.

The Alfa Data IDE driver might cause different types of trouble, like your partition problems, but I don't know.

If you want I can still have a look at the card and confirm if it works well on a standard setup or not.
It would be nice if you had another go with it and described the problems you had with it as precisely as possible.
 
@frodegh:

Sorry, didnt notice you replied to me :)

Here's the original thread of my problem:
https://www.amibay.com/showthread.php?t=14734

The situation today, after much tweaking & experimentation: copying doesnt get "stuck" as often, but the files are mostly corrupt, except very small files of a few kilobytes.
Even when trying to just read from the CF, without writing anywhere, the read data is corrupt (testing LHA/ZIP files nearly always fails).
This happens only on my A1200/060/PPC/OS3.9 configuration, on another 1200 with 030 all is well.

There was a suggestion in my original thread that it could be a HW problem, but since I only have a single A1200 board that works ok with my BPPC, I cant test this theory...
 
Thanks Uberfreak. So in addition to the problems described in your thread you have data corruption on reading the CF.
That should rule out the harddrive for now.

It might be a physical problem with the PCMCIA of your a1200.

There are some things you could try to narrow down the problem however:
- Try booting without startup-sequence, mount CF0, and check
- Try with WB3.0/3.1
- Try without BPPC
- Check that the CF mountlists (CF0 file) are identical on the two systems

You could try with the harddrive from the other a1200. (If I recall correctly my ClassicWB 3.1 installation also worked on kickstart 3.0)
Or you could even strip a wb-floppy, throw in compactflash.device and fat95, remove HDD and BPPC, boot from the floppy and copy from the CF to itself as a test.


You also said above that this was a common problem. Do you have any other references?
 
Last edited:
Are Kingston 8GB CF cards (with the flower on the front) suitable for Amiga CFHD usage? and is £11 a good price do you think for a brand new one?
 
Back
Top Bottom