2.6.40-rc1

2.6.40 development is on, and it's nice to see that it will include a number of MMC subsystem patches that I have written. It feels good to make a little difference.

Andrei Warkentin (17):
     mmc: Reliable write support.
     mmc: quirks: Extends card quirks with MMC/SD quirks matching the CID.
     mmc: core: Rename erase_timeout to cmd_timeout_ms.
     mmc: sdhci: R1B command handling + MMC_CAP_ERASE.
     mmc: core: Allow setting CMD timeout for CMD6 (SWITCH).
     mmc: card: block.c cleanup for host claim/release.
     mmc: MMC boot partitions support.
     mmc: quirks: Support for block quirks.
     mmc: quirks: Fix erase/trim for certain SanDisk cards.
     mmc: core: Fix use of uninitialized data in mmc_cmd_app.
     mmc: Ensure hardware partitions don't mess with mmcblk device naming.
     mmc: quirks: Add/remove quirks conditional support.
     mmc: core: Use CMD23 for multiblock transfers when we can.
     mmc: sdhci: Implement MMC_CAP_CMD23 for SDHCI.
     mmc: core: Block CMD23 support for UHS104/SDXC cards.
     mmc: sdhci: Auto-CMD23 support.
     mmc: sdhci: Auto-CMD23 fixes.

Overview, in no particular order:

• General refactoring of MMC block code.
• MMC block device hardware partition support, in particular implementing support for eMMC 4.3 hardware partitions. eSD partitions still not implemented due to lack of documentation in simplified spec.
• Due to the high potential of bricking a device, "unsafe" partitions like the boot partitions are marked read-only by default, and can be modified only after modifying the 'force_ro' sysfs parameter.
• Changes to the per-card quirks mechanism, which extends matching by CID/name/rev, and allows function-specific quirks tables, so block quirks don't have to pollute core/ code.
• Support for Sandisk eMMC TRIM/ERASE brain damage.
• Proper handling of R1B commands (allowing per-command timeouts).
• Support for CMD23-enabled multiblock transfers, which greatly (30% realistic) improves performance of some cards. Some cards have a slight decrease (unexplained by vendor), and so are blacklisted.
• SDHCI implementation of CMD23 support.
• SDHCI Auto-CMD23 support.
• Support for reliable writes. Exposed as REQ_FUA/REQ_META requests, which some people may find inappropriate, but there isn't a better match and I've yet to see better suggestions (more on this in a later post).

MMC block quirks.

So I'm secretly working on some block MMC quirks for some eMMC devices. It's not that big of a secret, but I'll keep the specifics to myself until I have collected enough data to post to linux-mmc. I already posted there the conclusions I reached based on other people's partial data, but that tends to be insufficient when it comes to convincing others of taking weird-looking code...

Anyway, the point is that occasionally you might wish the generic MMC code to act differently for various devices, whether as a workaround for not complying to the spec, or as a workaround for some smart proprietary intellectual property running inside these devices. The linux-next tree contains a quirks layer by Pierre Tardy, with a slight slant towards SDIO devices, as it was added to resolve some SDIO issue. Here is a patch against that tree that extends the quirk support to MMC/SD devices, allowing matching on CID fields. If any luck has it, it will be merged eventually. It would probably help if I hurried on the MMC block quirks actually using this ;-)... Data collection and analysis is a pain.

Measuring SQLite performance on MMC

I needed (still need to, gah) collect data on how a particular change to the Linux kernel MMC block layer ultimately affects write performance in SQLite. Android tends to use SQLite databases quite a bit, so a simple test on how long it takes to insert some large number of records is probably fair.

I exposed a couple of tunables through debugfs and started collecting data.

Of course, the first set of data was collected using specific synthetic benchmarks that did various kinds of O_SYNC+O_DIRECT I/O to the MMC, but it's nice to be able to see the real-life effect of a particular change, especially in combination with file systems. The problem is that you can get data, but that data is not repeatable within the same configuration. That makes comparing data across different configurations tedious at best.

Some things are pretty obvious - ensuring all mounted file systems are read-only. Ensuring there are no other processes running, etc. Other items don't come immediately to mind (to me at least :-()...

For example, running the same 5000 insertions x 50 times about 10 times on the same file system yields progressively slower data.

Current steps to ensure more-or-less repeatable data gathering:
0) Turn off CPU frequency scaling, RAM frequency scaling, etc, all
    power savings off.
1) Unmount partition containing files on which the SQLite test operates.
2) Perform BLKDISCARD over the partition.
3) Format with desired file system.
4) Mount.
6) Sync && echo 3 > /proc/sys/vm/drop_caches
7) Perform test.
8) Umount.
9) Repeat from (1)

Anything else I could have missed?