Go-libp2p Messages Concatenated Unexpectedly

Hi,

I’ve been noticing sometimes my messages get dropped. After some digging, I realized when messages get sent too closely together, sometimes they get concatenated into a single message (msg1 and msg2 get turned into msg1msg2). This is a problem because the multicodec unmarshalling ignores one of the messages.

I’ve written 2 examples to illustrate this issue.

In race1(), you will see the messages getting concatenated like this:

Got message: 01
Got message: 23
Got message: 45
Got message: 6
Got message: 7
Got message: 89
Got message: 1011
Got message: 1213
Got message: 14
Got message: 15
Got message: 16
Got message: 1718
Got message: 19

In race2(), you’ll see messages getting dropped:

Got message: 0
Got message: 2
Got message: 4
Got message: 5
Got message: 7
Got message: 9
Got message: 10
Got message: 12
Got message: 16
Got message: 18

I could try and do the unmarshalling manually, but the data off the wire is encrypted because I’m using encrypted connections, and I’m not quite sure how to decrypt it manually.

got /json
{"Op":0,"Data":"Of+BAwEBDVN0cmVhbURhdGFNc2cB/4IAAQMBBVNlcU5vAQYAAQZTdHJtSUQBDAABBERhdGEBCgAAAAb/ggMBMAA="}

got /json
{"Op":0,"Data":"Of+BAwEBDVN0cmVhbURhdGFNc2cB/4IAAQMBBVNlcU5vAQYAAQZTdHJtSUQBDAABBERhdGEBCgAAAAb/ggMBMQA="}
/json
{"Op":0,"Data":"Of+BAwEBDVN0cmVhbURhdGFNc2cB/4IAAQMBBVNlcU5vAQYAAQZTdHJtSUQBDAABBERhdGEBCgAAAAb/ggMBMgA="}

Anything I can do to make sure the messages aren’t concatenated?

Thanks,
Eric

This is working as expected. Streams are byte-oriented, not message oriented. Really, we should be concatenating more of these; we aren’t buffering nearly as much as we should be.

To fix this, you should wrap the byte-oriented stream in a message-oriented wrapper. Go protobuf provides one way to do this. If you don’t need something that fancy (you just want to send string messages, we have a simple library called go-msgio for sending length delineated messages. Alternatively, if you just want to send JSON, you can use go’s built-in JSON stream decoder encoding/json.

Edit: turns out the second part was a bug.

2 Likes