Written by Kishore Athrasseri
Published on Fri Oct 31 2025
I have been using MongoDB every day in my job for the last three and a half years. I’m fairly well acquainted with its rich query interface. The application I’m building uses a self-hosted mongod server, which means I also have a first-hand experience in maintaining a long-running MongoDB Community Edition server in production. This includes setting up replica set configuration, writing backup scripts, etc.
I love working with MongoDB as a database user and administrator, but so far hadn’t dived deeper into its internals.
One of these days I was thinking… What if I talk directly to the server without the client? Can I get the server to perform actions by directly sending commands over a TCP connection? On the other side, can I impersonate a server and talk to the client? Can I accept commands from the client and handle them in my own unique way, thereby implementing a custom MongoDB backend that can talk to any MongoDB client?
These are not original ideas. I’ve been aware of the MongoDB wire protocol and the possibility of custom backends ever since I came across FerretDB several months ago. However, it wasn’t something I had considered possible to do myself. Until recently, that is.
I decided to try.
I started an empty test database server.
user@host$ mkdir /tmp/mongo-data
user@host$ mongod --dbpath /tmp/mongo-dataConnecting to this server using MongoDB Compass, I created a new database called ‘app’ and an empty collection called ‘users’ inside the ‘app’ database. I wanted to try and insert a document into the users collection, not through Compass, but by directly sending a command to the server over the wire (TCP).
For this, I needed to find out what command should be sent for the server to execute a collection.insertOne(). The MongoDB official documentation has a page on the Wire Protocol that’s used in TCP communication between the server and client.
It was a bit intimidating in the beginning to make sense of, since I spend most of my time writing web application code and not so much at the level of binary communication.
One thing that stood out was that there used to be a variety of opcodes sent over the wire, such as OP_INSERT, OP_QUERY, OP_UPDATE, OP_DELETE, etc., but most of them have been deprecated in v5.0 in favour of a single OP_MSG that handles all operations. More versatile, but it meant digging through another layer of detail.
I realised that the second layer was nothing but the database commands already defined in the MongoDB shell. Having been involved in the maintenance of mongod servers, I was familiar with the practice of running commands in the shell, using the db.runCommand({ <command> }) syntax.
This was neat. I understood what the MongoDB authors were trying to do. Reduce the number of message formats on the wire to just one, and send the various commands wrapped in the same message format. I can imagine this approach must make it a lot easier to extend the repository in the future with a new command, rather than introducing and testing a new TCP message format for every new command.
There are two parts to crafting the message to be sent on the wire. First, the OP_MSG container needs to be created. Then, the insert command needs to be embedded in it.
The OP_MSG bytes are laid out as follows:
OP_MSG {
MsgHeader header;
uint32 flagBits;
Sections[] sections;
option<uint32> checksum;
}Source: MongoDB Wire Protocol
The OP_MSG message, like all wire messages, starts with a standard message header.
struct MsgHeader {
int32 messageLength;
int32 requestID;
int32 responseTo;
int32 opCode;
}Source: MongoDB Wire Protocol
messageLength is the number of bytes in the message including the message header. requestID can be any integer that we generate - the server will use requestID as a reference in the responseTo field when it replies. opCode is 2013 for OP_MSG. There are other values that signify the deprecated legacy opCodes, which we need not concern ourselves with at this point.
There are some flags that can be set to signal certain configuration settings to the server, but we don’t need any of them to test a simple insert command, so I’m going to simply set this field to 0.
This is the body of the message where the command will be embedded. There are two types of sections available and it’s an array field that can hold multiple sections, but for our insert command, we just need a simple ‘kind-0’ section. Which is nothing but a single byte with the value 0 (section kind) followed by the command encoded as BSON.
We will skip the optional checksum.
Now that we know how the message needs to be formatted, let’s look at the message content that we wish to send.
{
insert: <collection>,
documents: [ <document>, <document>, <document>, ... ],
ordered: <boolean>,
maxTimeMS: <integer>,
writeConcern: { <write concern> },
bypassDocumentValidation: <boolean>,
comment: <any>
}Source: MongoDB Wire Protocol
All fields exceptinsert and documents are optional, so we’ll skip them and keep our command as simple as we can.
{
insert: 'users',
documents: [
{
username: 'user1',
email: 'user1@example.org'
}
]
}We now know enough to start crafting the message. Let’s start with the sections. We will write a function that encodes the command document into a BSON buffer. I’m going to use Typescript.
import { BSON } from 'bson';
// We use the name 'sections' in plural, in case we want to
// extend the function to support kind-1 sections later. For now,
// we will accept only a single document as the body.
function encodeSections(commandDoc: any) {
const sectionKindByte = Buffer.alloc(1);
sectionKindByte.writeInt8(0);
const sections = BSON.serialize(commandDoc);
return Buffer.concat([sectionKindByte, sections]);
}
All we need to do to complete the message payload is to add the flag bits before the sections.
function encodePayload({
commandDoc,
flagBits,
}: {
commandDoc: any,
flagBits: number,
}): Buffer {
const flagBitsBuf = Buffer.alloc(4);
flagBitsBuf.writeInt32LE(flagBits);
const sectionsBuf = encodeSections(commandDoc);
return Buffer.concat([flagBitsBuf, sectionsBuf]);
}We now know the exact payload to be sent, and are ready to create the message header. Once that is done, our message will be complete.
function encodeOpMsg({
commandDoc,
flagBits,
}: {
commandDoc: any,
flagBits: number,
}): Buffer {
const payloadBuf = encodePayload({
commandDoc,
flagBits,
});
// Four int32 fields in the header
const messageHeaderBuf = Buffer.alloc(16);
const messageLength = messageHeaderBuf.length + payloadBuf.length;
messageHeaderBuf.writeInt32LE(messageLength, 0);
// requestID will be a dynamically generated reference in practice.
const requestID = 1;
messageHeaderBuf.writeInt32LE(requestID, 4);
// responseTo will usually be 0 for client-sent messages
const responseTo = 0;
messageHeaderBuf.writeInt32LE(responseTo, 8);
// opCode for OP_MSG is 2013
const opCode = 2013;
messageHeaderBuf.writeInt32LE(opCode, 12);
return Buffer.concat([messageHeaderBuf, payloadBuf]);
}
I’m going to skip implementing the TCP communication part in Typescript, and use a readily available utility like nc (netcat) to connect to the mongod server and send the command.
So we will instead invoke the encoder function and save the buffer to a binary file that we can redirect to the input of nc.
import fs from 'fs';
...
const commandDoc = {
insert: 'users',
documents: [
{
username: 'user1',
email: 'user1@example.org',
},
],
};
const opMsgBuf = encodeOpMsg({
commandDoc,
flagBits: 0,
});
fs.writeFileSync('insert.hex', opMsgBuf);Compiling and running the file creates a binary insert.hex file in the root directory of the project. This is the encoded OP_MSG message we’ll send to mongod over TCP.
user@host$ hexedit insert.hex
00000000 75 00 00 00 01 00 00 00 00 00 00 00 DD 07 00 00 u...............
00000010 00 00 00 00 00 60 00 00 00 02 69 6E 73 65 72 74 .....`....insert
00000020 00 06 00 00 00 75 73 65 72 73 00 04 64 6F 63 75 .....users..docu
00000030 6D 65 6E 74 73 00 3E 00 00 00 03 30 00 36 00 00 ments.>....0.6..
00000040 00 02 75 73 65 72 6E 61 6D 65 00 06 00 00 00 75 ..username.....u
00000050 73 65 72 31 00 02 65 6D 61 69 6C 00 12 00 00 00 ser1..email.....
00000060 75 73 65 72 31 40 65 78 61 6D 70 6C 65 2E 6F 72 user1@example.or
00000070 67 00 00 00 00 g....
Let’s send the command and see what happens.
user@host$ nc -N 127.0.0.1 27017 < insert.hex
I�jokerrmsg'OP_MSG requests require a $db argumentcode{�codeNameLocation40571user@host$The command was sent but it exited with an error which said that a $db argument was required. It makes sense, because when you execute a command from the MongoDB shell, it is done by default against a specific database as db.runCommand({ <command> }). It makes sense that when it is sent over the wire it needs to be specified in a separate $db field. However I couldn’t find this in the official documentation. All I could find was some discussion in some forum posts.
const commandDoc = {
insert: 'users',
documents: [
{
username: 'user1',
email: 'user1@example.org',
},
],
$db: 'app',
};This time there is a response from the server that looks like an ok reply.
user@host$ nc -N 127.0.0.1 27017 < insert.hex
-�nok�?user@host$And sure enough, refreshing Compass confirmed that the new user document has been inserted into the collection!
But I’m not fully satisfied with this. I should make it a point to figure out where this is documented.
A further quirk I discovered by accident is that the $db field should not be the first field in commandDoc, since mongod interprets the first key appearing in the encoded BSON as the command. Again, this doesn’t seem to be documented.
This rounds up a successful attempt to talk to mongod without the help of a client or a driver. The protocol seems simple enough. Of course, there are several commands that would need to be implemented to support a full client-server conversation. Perhaps even some of the legacy opcodes.
The next step in that direction is to understand the sequence of messages passed between server and client during the normal course of operation. That will give us a sense of the minimal functionality needed in a custom backend to convince an unsuspecting MongoDB client that it is indeed talking to mongod.
*This exploration is Part 1 of the journey in building SQLite-Mongo, a mongod-compatible server with an SQLite backend. The full code for this exploration can be found here.