MongoDB Experience: Replication 101

After successfully installing and testing mongoDB it’s very easy to create a replication environment.

$ mkdir -p data/{master,slave}
$ mongod --dbpath=`pwd`/data/master --master --port 28011 > master.log 2>&1 &
# Always check your log file
$ cat master.log
$ mongod --dbpath=`pwd`/data/slave --slave --source localhost:28011 --port 28022 > slave.log 2>&1 &
$ cat slave.log

The options are relatively descriptive and straightforward.

  • –dbpath – The directory for data (we set because we are running master/slave on same server)
  • –port – Likewise we are running multiple instances on same machine
  • –master – I’m the master
  • –slave – I’m a slave
  • –source – For slaves, tell them were the source (i.e. master is)

What I found under the covers was a difference from the single instance version. There is a series of ‘local’ files for the namespace, where in the single instance version there were ‘test’ files.

$ ls -ltR data
total 0
drwxr-xr-x  6 rbradfor  staff  204 Jun 10 10:24 slave
drwxr-xr-x  5 rbradfor  staff  170 Jun 10 10:22 master

data/slave:
total 163848
drwxr-xr-x  2 rbradfor  staff        68 Jun 10 10:24 _tmp
-rw-------  1 rbradfor  staff  67108864 Jun 10 10:24 local.0
-rw-------  1 rbradfor  staff  16777216 Jun 10 10:24 local.ns
-rwxr-xr-x  1 rbradfor  staff         6 Jun 10 10:24 mongod.lock

data/slave/_tmp:

data/master:
total 163848
-rw-------  1 rbradfor  staff  67108864 Jun 10 10:22 local.0
-rw-------  1 rbradfor  staff  16777216 Jun 10 10:22 local.ns
-rwxr-xr-x  1 rbradfor  staff         6 Jun 10 10:22 mongod.lock

A quick replication test.

$ mongo --port 28011
MongoDB shell version: 1.4.3
url: /blogtest
connecting to: 127.0.0.1:28011/test
type "help" for help
> db.foo.save({s:"Hello world"});
> db.foo.find();
{ "_id" : ObjectId("4c10f7904a30c35548b0af06"), "s" : "Hello world" }
> exit
bye

$ mongo --port 28022
MongoDB shell version: 1.4.3
url: /blogtest
connecting to: 127.0.0.1:28022/test
type "help" for help
> db.foo.find();
{ "_id" : ObjectId("4c10f7904a30c35548b0af06"), "s" : "Hello world" }
> exit

A look now at the underlying data shows a ‘test’ namespace which confirms the lazy instantiation approach. The ‘local’ namespace files are obviously a reflection of the –master/–slave operation.

$ ls -ltR data
total 0
drwxr-xr-x  9 rbradfor  staff  306 Jun 10 10:32 slave
drwxr-xr-x  8 rbradfor  staff  272 Jun 10 10:32 master

data/slave:
total 589832
-rw-------  1 rbradfor  staff  134217728 Jun 10 10:33 test.1
drwxr-xr-x  2 rbradfor  staff         68 Jun 10 10:32 _tmp
-rw-------  1 rbradfor  staff   67108864 Jun 10 10:32 test.0
-rw-------  1 rbradfor  staff   16777216 Jun 10 10:32 test.ns
-rw-------  1 rbradfor  staff   67108864 Jun 10 10:24 local.0
-rw-------  1 rbradfor  staff   16777216 Jun 10 10:24 local.ns
-rwxr-xr-x  1 rbradfor  staff          6 Jun 10 10:24 mongod.lock

data/master:
total 327688
drwxr-xr-x  2 rbradfor  staff        68 Jun 10 10:32 _tmp
-rw-------  1 rbradfor  staff  67108864 Jun 10 10:32 test.0
-rw-------  1 rbradfor  staff  16777216 Jun 10 10:32 test.ns
-rw-------  1 rbradfor  staff  67108864 Jun 10 10:22 local.0
-rw-------  1 rbradfor  staff  16777216 Jun 10 10:22 local.ns
-rwxr-xr-x  1 rbradfor  staff         6 Jun 10 10:22 mongod.lock

By default there appears to be no read-only default state for a slave. I was able to add new data to the slave.

$ mongo --port 28022
MongoDB shell version: 1.4.3
url: /blogtest
connecting to: 127.0.0.1:28022/test
type "help" for help
> db.foo.save({s:"Hello New York"});
> db.foo.find();
{ "_id" : ObjectId("4c10f7904a30c35548b0af06"), "s" : "Hello world" }
{ "_id" : ObjectId("4c10f864d8e80f1a1ad305cf"), "s" : "Hello New York" }
>

A closer look at this ‘local’ namespace and a check via the docs gives us details of the slave configuration.

$ mongo --port 28022
MongoDB shell version: 1.4.3
url: /blogtest
connecting to: 127.0.0.1:28022/test
type "help" for help
> show dbs;
admin
local
test
> use local;
switched to db local
> show collections;
oplog.$main
pair.sync
sources
system.indexes
> db.sources.find();
{ "_id" : ObjectId("4c10f5b633308f7c3d7afc45"), "host" : "localhost:28011", "source" : "main", "syncedTo" : { "t" : 1276180895000, "i" : 1 }, "localLogTs" : { "t" : 1276180898000, "i" : 1 } }

You can also with the mongo client connect directly to a collection via the command line.

$ mongo localhost:28022/local
MongoDB shell version: 1.4.3
url: /bloglocalhost:28022/local
connecting to: localhost:28022/local
type "help" for help
> db.sources.find();
{ "_id" : ObjectId("4c10f5b633308f7c3d7afc45"), "host" : "localhost:28011", "source" : "main", "syncedTo" : { "t" : 1276180775000, "i" : 1 }, "localLogTs" : { "t" : 1276180778000, "i" : 1 } }
> exit
bye

The shell gives 3 convenience commands for showing replication state.

On the Slave

$ mongo --port 28022
> db.getReplicationInfo();
{
	"logSizeMB" : 50,
	"timeDiff" : 1444,
	"timeDiffHours" : 0.4,
	"tFirst" : "Thu Jun 10 2010 10:24:54 GMT-0400 (EDT)",
	"tLast" : "Thu Jun 10 2010 10:48:58 GMT-0400 (EDT)",
	"now" : "Thu Jun 10 2010 10:48:59 GMT-0400 (EDT)"
}
> db.printReplicationInfo();
configured oplog size:   50MB
log length start to end: 1444secs (0.4hrs)
oplog first event time:  Thu Jun 10 2010 10:24:54 GMT-0400 (EDT)
oplog last event time:   Thu Jun 10 2010 10:48:58 GMT-0400 (EDT)
now:                     Thu Jun 10 2010 10:49:07 GMT-0400 (EDT)
> db.printSlaveReplicationInfo();
source:   localhost:28011
syncedTo: Thu Jun 10 2010 10:49:25 GMT-0400 (EDT)
          = 1secs ago (0hrs)

On the master, the same commands are applicable, output basically the same.

$ mongo --port 28011
> db.getReplicationInfo();
{
	"logSizeMB" : 50,
	"timeDiff" : 1714,
	"timeDiffHours" : 0.48,
	"tFirst" : "Thu Jun 10 2010 10:22:01 GMT-0400 (EDT)",
	"tLast" : "Thu Jun 10 2010 10:50:35 GMT-0400 (EDT)",
	"now" : "Thu Jun 10 2010 10:50:40 GMT-0400 (EDT)"
}
> db.printReplicationInfo();
configured oplog size:   50MB
log length start to end: 1714secs (0.48hrs)
oplog first event time:  Thu Jun 10 2010 10:22:01 GMT-0400 (EDT)
oplog last event time:   Thu Jun 10 2010 10:50:35 GMT-0400 (EDT)
now:                     Thu Jun 10 2010 10:50:45 GMT-0400 (EDT)
> db.printSlaveReplicationInfo();
local.sources is empty; is this db a --slave?
>

From these commands there seems no obvious way to easily identify if an instance is a master or not.

References

DBA operations from shell
Replication
Master/Slave Replication

Tagged with: Mongodb Uncategorized

Related Posts

Mongodb

MongoDB Experience: Gotcha with collection names

In my earlier tests I bulk loaded data with the following command. mongoimport -d olympics -c olympic_event -type tsv --headerline -f name,id,sport,demonstration_competitions,olympic_games_contested,competitions,contested_as_demonstration_event --drop olympic_event.tsv connected to: 127.0.0.1 dropping: olympics.olympic_event imported 775 objects As you can see I imported 775 objects, however when I went to review them via the mongo interactive shell I found no data.

Read more
Mongodb

MongoDB Experience: What’s running in the DB

You can very easily find out the running threads in the database (e.g. like a MySQL SHOW PROCESSLIST) with db.currentOp . > db.currentOp(); { "inprog" : [ ] } No much happening, however under some load you can see

Read more
Mongodb

MongoDB Experience: Bulk Loading Data

mongoDB has a mongoimport command. The docs only shows the usage but not any examples. here are my first examples. data1.csv 1 2 3 4 5 6 7 8 9 0 You need to specify your database (-d) and collection (-c) for importing.

Read more