Send/Receiving big resources

Pelion Device Management [forum moved]
1

Hi

I am looking at how to send and receive large resources without having to load the complete message into RAM.

For receiving messages my approach is to receive each block and store it in a temporary nvs space then resemble once it we have received all the blocks. To achieve this we are setting the M2MResourceInstance external_blockwise to true and enabling a callback on set_incoming_block_message_callback to handle the incoming block.

Can you confirm if this a reliable mechanism, our testing so far has been reliable and however we have always received message blocks in order.

On the publishing side, I’m at a complete loss. The callbacks are not being called, I’m unsure whether setting the resource value is right approach because it always expects a pointer to a buffer value or value to be passed.

M2MObject* big_object = M2MInterfaceFactory::create_object("32771");
M2MObjectInstance* big_object_inst = big_object->create_object_instance();
M2MResource* big_res = big_object_inst->create_dynamic_resource("26244", "BigRes", M2MResourceInstance::STRING, true, false, true);

big_res->set_value((uint8_t*)"0", 1); // unsure what this for ?
big_res->set_operation(M2MBase::GET_PUT_ALLOWED);

big_res->set_register_uri(true);
big_res->publish_value_in_registration_msg(false);
big_res->set_auto_observable(true);
big_res->set_max_age(3);

big_res->set_message_delivery_status_cb(
  (void (*)(const M2MBase&,
  const M2MBase::MessageDeliveryStatus,
  const M2MBase::MessageType,
  void*))(void*)Cloud::resource_published_cb,
  NULL);

big_res->set_incoming_block_message_callback(incoming_block_message_callback(this, &Cloud::block_message_received));
big_res->set_outgoing_block_message_callback(outgoing_block_message_callback(this, &Cloud::block_request));

objectList.push_back(big_object);

Any ideas would be much appreciated. I have tried in vein examining the source for mbed-cloud-client to see how it works to try my best to get a better understanding. :persevere: I have also looked at the example mbed-os-example-client However this has been deprecated.

Thank you,
Ash

2

After some time getting familiar with the mbed-cloud-client source. I believe this not to be possible in the current version (4.3.0). My understanding is that this is due to the underlaying implementation in mbed-coap whereby it expects a location to the entire resource value, and it will handle responding to the subsequent block message requests. It would be nice if we were also able to pass down a callback function to provide the payload for other the blocks.

My conclusion is that it is NOT possible to stream a resource body where the size is larger than the available ram. This limitation should be taken into consideration in any future system designs.

As reference the closest I got was with some wally polly code 
M2MObject* big_object = M2MInterfaceFactory::create_object("32771");
M2MObjectInstance* big_object_inst = big_object->create_object_instance();
M2MResource* big_res = big_object_inst->create_dynamic_resource("26244", "BigRes", M2MResourceInstance::OPAQUE, true, false, true);

big_res->set_operation(M2MBase::GET_ALLOWED);

big_res->set_register_uri(true);
big_res->publish_value_in_registration_msg(false);
big_res->set_auto_observable(true);
big_res->set_max_age(3);

// incoming_block_msg_cb must be set in order to initialise _block_message_data on the resource
big_res->set_incoming_block_message_callback(incoming_block_message_callback(this, &Cloud::block_message_received));
big_res->set_outgoing_block_message_callback(outgoing_block_message_callback(this, &Cloud::block_request));

sn_coap_options_list_s options_list_ptr;
coapHeader.options_list_ptr = &options_list_ptr;
coapHeader.options_list_ptr->block1 = 0;
coapHeader.options_list_ptr->use_size1 = true; // trick the resource so that it originally received as a block message even though it didn't
coapHeader.options_list_ptr->size1 = 1024;// ensure this is bigger than specified SN_COAP_MAX_BLOCKWISE_PAYLOAD_SIZE
big_res->block_message()->set_message_info(&coapHeader);

Thanks,
Ash