SDBlockDevice performance issue

[Unturned3 (Richard Hu)]

Hello!

I’m making a battery-powered device that records ECG data and stores it onto an SD card. To minimize the time that the SD card is active, I’m trying to write data to it in periodic bursts. However, it seems like the write operations to the SD card is quite inefficient / slow. With the SCLK frequency set to 1Mhz, I only get about 27.7 kilobytes per second at most.

I checked the SPI bus activity with a logic analyzer (image attached below), and found that around half the time MOSI does nothing but transmit 0xFF to the SD card. May I ask what is the purpose of these 0xFF bytes? How can I improve the write speed to the SD card, aside from increasing the SCLK frequency?

Here is the code that I used. The uint8_t buf[8192] array stores the dummy data waiting to be written to the SD card. I’ve tried various sizes for the buffer, e.g. 512, 2048, 8192, but it only seems to improve the SD write data rate marginally.

#include <mbed.h>
#include <SDBlockDevice.h>
#include <FATFileSystem.h>

using namespace std::chrono;

UnbufferedSerial pc(USBTX, USBRX, 115200);		// debug serial port
FileHandle *mbed::mbed_override_console(int fd){ return &pc; }

SDBlockDevice sd(PB_5, PB_4, PB_3, PB_1, 1000000);
FATFileSystem fs("sd_card");

uint8_t buf[8192];

Timer T;

int main(){
	fs.mount(&sd);

	for(int i=0; i<8192; i++)
		buf[i] = (uint8_t)i;	// Create the dummy data

	T.start();	// Start timer

	FILE *fp = fopen("/sd_card/test3", "ab");

	for(int i=0; i<512; i++)	// Write 4 megabytes of data
		fwrite(buf, 1, 8192, fp);

	fflush(fp);
	T.stop();	// Stop timer
	fclose(fp);
	printf("Time (ms): %llu\r\n", duration_cast<milliseconds>(T.elapsed_time()).count());
	return 0;
}

Screen shot from logic analyzer:

Thanks in advance for helping!

[hudakz (Zoltan Hudak)]

Hello Richard,

@Unturned3

I checked the SPI bus activity with a logic analyzer (image attached below), and found that around half the time MOSI does nothing but transmit 0xFF to the SD card. May I ask what is the purpose of these 0xFF bytes?

In first step Mbed initializes the SD card at low SPI frequency (generally in the 100 kHz - 400 kHz range) by transmitting 0xFF over the MOSI line. According to your description this took about half of the time. It seems that the initialization frequency selected by Mbed is not suitable for your SD card. To test other frequencies you can set it manually over the mbed_app.json file, for example as follows:

{
    "target_overrides": {
        "*": {
            "sd.INIT_FREQUENCY": 160000
        }
    }
}

I run your code on an LPC1768 board with an 8GB Class 6 SD card and the speed was about 83 kilobytes per second.

[Unturned3 (Richard Hu)]

Hello Zoltan,

Thanks for the help! I’ve tried your suggestion to change the initialization frequency, but there weren’t any changes to the data transmission rate.

I’m not too sure when exactly is the SD card initialized (using the low SPI frequency). Is the SD initialized once when it is powered-on, or does this initialization process happen every time the SD card is activated by the CS pin?

Just to clarify, when I said “around half the time MOSI does nothing but transmit 0xFF”, I was actually referring to the sections highlighted by the blue rectangles in the image below. I don’t think these sections are the SD initialization transmissions, because I zoomed in on the SCLK line in the logic analyzer and saw all the pulses were still around 1Mhz, not in the range of 100 to 400KHz.

I’m expecting to see a more or less continuous transfer of data on the MOSI lines, instead of it having many “gaps” of 0xFF bytes (or is this an incorrect expectation?)

[hudakz (Zoltan Hudak)]

@Unturned3

I’m not too sure when exactly is the SD card initialized (using the low SPI frequency). Is the SD initialized once when it is powered-on, or does this initialization process happen every time the SD card is activated by the CS pin?

The SD card is initialized by calling the SDBlockDevice::init() method (member function). In your program this happens when mounting the card.

@Unturned3

Just to clarify, when I said “around half the time MOSI does nothing but transmit 0xFF”, I was actually referring to the sections highlighted by the blue rectangles in the image below. I don’t think these sections are the SD initialization transmissions, because I zoomed in on the SCLK line in the logic analyzer and saw all the pulses were still around 1Mhz, not in the range of 100 to 400KHz.

You are right. Those section are not SD initialization transmissions. I figured that the SDBlockDevice::_cmd_spi function transmits 0xFF in command packet at position 6 by default when CRC is off:

uint8_t SDBlockDevice::_cmd_spi(SDBlockDevice::cmdSupported cmd, uint32_t arg)
{
    uint8_t response;
    char cmdPacket[PACKET_SIZE];

    // Prepare the command packet
    cmdPacket[0] = SPI_CMD(cmd);
    cmdPacket[1] = (arg >> 24);
    cmdPacket[2] = (arg >> 16);
    cmdPacket[3] = (arg >> 8);
    cmdPacket[4] = (arg >> 0);

#if MBED_CONF_SD_CRC_ENABLED
    if (_crc_on) {
        MbedCRC<POLY_7BIT_SD, 7> crc7;
        uint32_t crc;
        crc7.compute(cmdPacket, 5, &crc);
        cmdPacket[5] = ((uint8_t) crc << 1) | 0x01;
    } else
#endif
    {
        // CMD0 is executed in SD mode, hence should have correct CRC
        // CMD8 CRC verification is always enabled
        switch (cmd) {
            case CMD0_GO_IDLE_STATE:
                cmdPacket[5] = 0x95;
                break;
            case CMD8_SEND_IF_COND:
                cmdPacket[5] = 0x87;
                break;
            default:
                cmdPacket[5] = 0xFF;    // Make sure bit 0-End bit is high
                break;
        }
    }

    // send a command
    for (int i = 0; i < PACKET_SIZE; i++) {
        _spi.write(cmdPacket[i]);
    }

    // The received byte immediataly following CMD12 is a stuff byte,
    // it should be discarded before receive the response of the CMD12.
    if (CMD12_STOP_TRANSMISSION == cmd) {
        _spi.write(SPI_FILL_CHAR);
    }

    // Loop for response: Response is sent back within command response time (NCR), 0 to 8 bytes for SDC
    for (int i = 0; i < 0x10; i++) {
        response = _spi.write(SPI_FILL_CHAR);
        // Got the response
        if (!(response & R1_RESPONSE_RECV)) {
            break;
        }
    }
    return response;
}

In order to to get debug messages and enable command tracing open the mbed-os/storage/blockdevice/COMPONENT_SD/include/SD/SDBlockDevice.cpp file and set:

...
#define SD_DBG                                   1      /*!< 1 - Enable debugging */
#define SD_CMD_TRACE                             1      /*!< 1 - Enable SD command tracing */
...

Then rebuild and run your program.

[Unturned3 (Richard Hu)]

Hello Zoltan,

I read the code in SDBlockDevice.cpp, and found a function called SDBlockDevice::_wait_ready that seems to continuously send 0xFF bytes over MOSI until the MISO line transitions from low to high (i.e. when a 0xFF is sent by the SD card). The SDBlockDevice::_write function invokes also _wait_ready, so I suspect _wait_ready may be the cause of the blue sections in the logic analyzer image above.

To verify this, I defined a DigitalOut pin called DOP, and every time _write invokes _wait_ready the pin would be set high (and reset to low once _wait_ready finishes).

It does seem like that a good portion of the transmission time is spent doing nothing but running _wait_ready for the SD card. Why does the SD card need to wait so frequently? Could the number of waits be lowered by somehow sending data in “bulk”?

I also tried your suggestion of enabling SD_DBG and SD_CMD_TRACE, and obtained the following:

CMD:0       arg:0x0         Response:0x1
CMD:8    arg:0x1aa       Response:0x1
V2-Version Card
R3/R7: 0x1aa
CMD:58   arg:0x0         Response:0x1
R3/R7: 0xff8000
CMD:41   arg:0x40000000          Response:0x1
CMD:41   arg:0x40000000          Response:0x0
CMD:58   arg:0x0         Response:0x0
R3/R7: 0xc0ff8000
Card Initialized: High Capacity Card
CMD:59   arg:0x0         Response:0x0
init card = 1
CMD:9    arg:0x0         Response:0x0
SDHC/SDXC Card: hc_c_size: 59637
Sectors: 0x3a3d800x : 61069312
Capacity: 29819 MB
CMD:16   arg:0x200       Response:0x0
CMD:17   arg:0x0         Response:0x0
CMD:17   arg:0x2         Response:0x0
CMD:17   arg:0x3         Response:0x0
CMD:17   arg:0x7490      Response:0x0
CMD:24   arg:0x7490      Response:0x0
CMD:17   arg:0x31        Response:0x0
CMD:24   arg:0x31        Response:0x0
CMD:24   arg:0x3a68      Response:0x0
CMD:17   arg:0x32        Response:0x0
CMD:24   arg:0x32        Response:0x0
CMD:24   arg:0x3a69      Response:0x0
CMD:17   arg:0x33        Response:0x0
CMD:32   arg:0x16bb0     Response:0x0
CMD:33   arg:0x18baf     Response:0x0
CMD:38   arg:0x0         Response:0x0
CMD:24   arg:0x33        Response:0x0
CMD:24   arg:0x3a6a      Response:0x0
CMD:17   arg:0x7490      Response:0x0
CMD:17   arg:0x31        Response:0x0
CMD:23   arg:0x2         Response:0x0
...

I am not sure how to interpret this. The card initialization steps shown by the trace output seems normal to me.

Thank you.

[hudakz (Zoltan Hudak)]

…
Card Initialized: High Capacity Card
…
Capacity: 29819 MB

You seems to have a high capacity SD card. I’m not sure Mbed OS 6 can handle it correctly.
To verify, try to run this Mbed 2 program (load it to the online compiler, don’t change anything, just compile for your target) and check the speed.

[Unturned3 (Richard Hu)]

Hello Zoltan,

When I imported the Mbed 2 program into the online compiler, a window popped up saying “an error occurred when importing repository”, although no additional details were shown.

When I tried to compile it, I got an the following message: "Error: Fatal error: C3903U: Argument ‘Cortex-M4.fp.sp’ not permitted for option ‘cpu’. " No additional details are shown. I assume this is an error related to a flag passed to the compiler?

[hudakz (Zoltan Hudak)]

You haven’t told us the target board name. Mbed 2 supports:

ARCH_BLE,              ARCH_BLE_BOOT,         ARCH_BLE_OTA, 
ARCH_GPRS,             ARCH_LINK,             ARCH_LINK_BOOT, 
ARCH_LINK_OTA,         ARCH_MAX,              ARCH_PRO, 
ARM_BEETLE_SOC,        ARM_CM3DS_MPS2,        ARM_IOTSS_BEID, 
ARM_MPS2_M0,           ARM_MPS2_M0P,          ARM_MPS2_M3, 
ARM_MPS2_M4,           ARM_MPS2_M7,           B96B_F446VE, 
BLACKPILL,             BLUEPILL,              BLUEPILL_F103C8, 
CY8CKIT_062_4343W,     CY8CKIT_062_BLE,       CY8CKIT_062_WIFI_BT, 
CY8CPROTO_062_4343W,   CYW943012P6EVB_01,     DELTA_DFBM_NQ620, 
DELTA_DFCM_NNN40,      DELTA_DFCM_NNN40_BOOT, DELTA_DFCM_NNN40_OTA, 
DELTA_DFCM_NNN50,      DELTA_DFCM_NNN50_BOOT, DELTA_DFCM_NNN50_OTA, 
DISCO_F051R8,          DISCO_F100RB,          DISCO_F303VC, 
DISCO_F334C8,          DISCO_F401VC,          DISCO_F407VG, 
DISCO_F413ZH,          DISCO_F429ZI,          DISCO_F469NI, 
DISCO_F746NG,          DISCO_F769NI,          DISCO_L053C8, 
DISCO_L072CZ_LRWAN1,   DISCO_L475VG_IOT01A,   DISCO_L476VG, 
DISCO_L496AG,          EFM32GG11_STK3701,     EFM32GG_STK3700, 
EFM32HG_STK3400,       EFM32LG_STK3600,       EFM32PG12_STK3402, 
EFM32PG_STK3401,       EFM32WG_STK3800,       EFM32ZG_STK3200, 
ELEKTOR_COCORICO,      ELMO_F411RE,           EV_COG_AD3029LZ, 
EV_COG_AD4050LZ,       FF1705_L151CC,         FF_LPC546XX, 
FUTURE_SEQUANA,        FUTURE_SEQUANA_M0,     FUTURE_SEQUANA_M0_PSA, 
FUTURE_SEQUANA_PSA,    FVP_MPS2_M0,           FVP_MPS2_M0P, 
FVP_MPS2_M3,           FVP_MPS2_M4,           FVP_MPS2_M7, 
GD32_E103VB,           GD32_F307VG,           GD32_F450ZI, 
GR_LYCHEE,             HEXIWEAR,              HRM1017, 
HRM1017_BOOT,          HRM1017_OTA,           K20D50M, 
K22F,                  K64F,                  K66F, 
K82F,                  KL05Z,                 KL25Z, 
KL26Z,                 KL27Z,                 KL43Z, 
KL46Z,                 KL82Z,                 KW24D, 
KW41Z,                 LPC1114,               LPC11C24, 
LPC11U24,              LPC11U24_301,          LPC11U34_421, 
LPC11U35_401,          LPC11U35_501,          LPC11U35_501_IBDAP, 
LPC11U35_Y5_MBUG,      LPC11U37H_401,         LPC11U37_501, 
LPC11U68,              LPC1347,               LPC1549, 
LPC1768,               LPC1768A,              LPC1769, 
LPC4088,               LPC4088_DM,            LPC4330_M0, 
LPC4330_M4,            LPC4337,               LPC54114, 
LPC546XX,              LPC810,                LPC812, 
LPC824,                LPCCAPPUCCINO,         LPCMINI, 
MAPLEMINI,             MAX32600MBED,          MAX32620FTHR, 
MAX32620HSP,           MAX32625MBED,          MAX32625NEXPAQ, 
MAX32625PICO,          MAX32630FTHR,          MAXWSNENV, 
MBED_CONNECT_ODIN,     MCU_LPC4088,           MICRONFCBOARD, 
MIMXRT1050_EVK,        MOTE_L152RC,           MTB_ACONNO_ACN52832, 
MTB_ADV_WISE_1510,     MTB_ADV_WISE_1530,     MTB_ADV_WISE_1570, 
MTB_LAIRD_BL600,       MTB_LAIRD_BL652,       MTB_LAIRD_BL654, 
MTB_MTS_DRAGONFLY,     MTB_MTS_XDOT,          MTB_MURATA_ABZ, 
MTB_MURATA_WSM_BL241,  MTB_MXCHIP_EMW3166,    MTB_RAK811, 
MTB_UBLOX_NINA_B1,     MTB_UBLOX_ODIN_W2,     MTB_USI_WM_BN_BM_22, 
MTM_MTCONNECT04S,      MTM_MTCONNECT04S_BOOT, MTM_MTCONNECT04S_OTA, 
MTS_DRAGONFLY_F411RE,  MTS_DRAGONFLY_L471QG,  MTS_GAMBIT, 
MTS_MDOT_F405RG,       MTS_MDOT_F411RE,       NCS36510, 
NRF51822,              NRF51822_BOOT,         NRF51822_OTA, 
NRF51822_Y5_MBUG,      NRF51_DK,              NRF51_DK_BOOT, 
NRF51_DK_LEGACY,       NRF51_DK_OTA,          NRF51_DONGLE, 
NRF51_DONGLE_BOOT,     NRF51_DONGLE_LEGACY,   NRF51_DONGLE_OTA, 
NRF51_MICROBIT,        NRF51_MICROBIT_B,      NRF51_MICROBIT_BOOT, 
NRF51_MICROBIT_B_BOOT, NRF51_MICROBIT_B_OTA,  NRF51_MICROBIT_OTA, 
NRF52840_DK,           NRF52_DK,              NUCLEO_F030R8, 
NUCLEO_F031K6,         NUCLEO_F042K6,         NUCLEO_F070RB, 
NUCLEO_F072RB,         NUCLEO_F091RC,         NUCLEO_F103RB, 
NUCLEO_F207ZG,         NUCLEO_F302R8,         NUCLEO_F303K8, 
NUCLEO_F303RE,         NUCLEO_F303ZE,         NUCLEO_F334R8, 
NUCLEO_F401RE,         NUCLEO_F410RB,         NUCLEO_F411RE, 
NUCLEO_F412ZG,         NUCLEO_F413ZH,         NUCLEO_F429ZI, 
NUCLEO_F439ZI,         NUCLEO_F446RE,         NUCLEO_F446ZE, 
NUCLEO_F746ZG,         NUCLEO_F756ZG,         NUCLEO_F767ZI, 
NUCLEO_H743ZI,         NUCLEO_L011K4,         NUCLEO_L031K6, 
NUCLEO_L053R8,         NUCLEO_L073RZ,         NUCLEO_L152RE, 
NUCLEO_L432KC,         NUCLEO_L433RC_P,       NUCLEO_L476RG, 
NUCLEO_L486RG,         NUCLEO_L496ZG,         NUCLEO_L496ZG_P, 
NUCLEO_L4R5ZI,         NUCLEO_L4R5ZI_P,       NUMAKER_IOT_M487, 
NUMAKER_PFM_M2351,     NUMAKER_PFM_M453,      NUMAKER_PFM_M487, 
NUMAKER_PFM_NANO130,   NUMAKER_PFM_NUC472,    NZ32_SC151, 
OC_MBUINO,             OSHCHIP,               RAPIDIOT_K64F, 
RAPIDIOT_KW41Z,        RBLAB_BLENANO,         RBLAB_BLENANO2, 
RBLAB_BLENANO_BOOT,    RBLAB_BLENANO_OTA,     RBLAB_NRF51822, 
RBLAB_NRF51822_BOOT,   RBLAB_NRF51822_OTA,    RDA5981X, 
REALTEK_RTL8195AM,     RO359B,                RZ_A1H, 
RZ_A1XX,               SAKURAIO_EVB_01,       SAMD21G18A, 
SAMD21J18A,            SAMG55J19,             SAML21J18A, 
SAMR21G18A,            SARA_NBIOT_EVK,        SDT32620B, 
SDT32625B,             SDT51822B,             SDT52832B, 
SDT64B,                SEEED_TINY_BLE,        SEEED_TINY_BLE_BOOT, 
SEEED_TINY_BLE_OTA,    SILICA_SENSOR_NODE,    SSCI824, 
STEVAL_3DP001V1,       STM32F103RC,           STM32F103VE, 
STM32F407VE,           TB_SENSE_1,            TB_SENSE_12, 
TEENSY3_1,             TMPM066,               TMPM3H6, 
TMPM3HQ,               TMPM46B,               TMPM4G9, 
TT_M3HQ,               TY51822R3,             TY51822R3_BOOT, 
TY51822R3_OTA,         UBLOX_C027,            UBLOX_C030_N211, 
UBLOX_C030_R410M,      UBLOX_C030_R412M,      UBLOX_C030_R41XM, 
UBLOX_C030_U201,       UBLOX_EVA_NINA,        UBLOX_EVK_NINA_B1, 
UBLOX_EVK_ODIN_W2,     UBRIDGE,               UNO_91H, 
USENSE,                VBLUNO51,              VBLUNO51_BOOT, 
VBLUNO51_LEGACY,       VBLUNO51_OTA,          VBLUNO52, 
VK_RZ_A1H,             WALLBOT_BLE,           WALLBOT_BLE_BOOT, 
WALLBOT_BLE_OTA,       WIO_3G,                WIO_BG96, 
WIZWIKI_W7500,         WIZWIKI_W7500ECO,      WIZWIKI_W7500P, 
XADOW_M0,              XBED_LPC1768,          XDOT_L151CC

[Unturned3 (Richard Hu)]

Hello Zoltan,
I’m using the Nucleo-L432KC board. Let me try to compile it again…

Edit: I selected “Update all libraries” when importing, and it seems to have fixed the “C3903U…” error.

However, the compiler then complained about not finding FATFileSystem.h. It seems like the FATFileSystem.lib file was removed for some reason when I imported the repository into the online compiler. I tried to manually import FATFileSystem from the library import wizard, but the compilation failed with error Error: Object of abstract class type "mbed::FATFileHandle" is not allowed in "FatFileSystem/FATFileSystem.cpp".