小塌客

石头的博客

Category "MongoDB"

MongoDB Sharding设计

这篇文章是shitou在目前的company(外企)的wiki上写的,是英文的,主要是关于mongodb sharding的设计以及主要问题的考虑,过于细节和基础的问题在这里并没有涉及,可以参考mongodb的官方文档

 

 

Sharding Key

Most of our queries are based monitor_id, timestamp and location, and according to the sharding key design principle: 

  • Single read should be finished in one shard.
  • All reads should be distributed to all shards.
  • Writes should be distributed to all shards.

so the sharding key will be: {xxx_id: 1, xxx: 1, xxx: 1}, Remember the sequence of it when creating index, the sequence is extremely important, because database index follows the LEFT MATCH principle.

 

Query

Most queries will hit the sharing key, however, we can not cover all situations and requirements are changing, we can not change the sharding key when we create it in database. When query not hit the sharding key, it will scan all the shards, it will be slower than hitting the sharding key query, but it is acceptable, we don't have too much resource to do this benchmark in this situation, but in mongodb official says, it will be still fast when the shards servers is under 10.

When non-sharing-key query scans all shards we need to make sure that it will still hit the index, otherwise, it will be definitely vey slow.

 

Chunks

The default chunk size is 64MB, when one chunk is bigger than that number, mongodb will split the chunk into two chunks, and move it to another shard.

  • If the number is too small, which will make mongodb be busy splitting chunks, especially when our data grow fast.
  • If the number is too big, it will make mongodb cost too much time to move chunks to another server.

we will use the default number in the beginning, and then to see if we need to increase the number or decrease the number.

 

When mongodb move chunks from one shard to another shard, the server load will grow, which maybe will cause performance problems, but we can control the chunk moving process, it's the balancer process, for example, if we notice that every time chunk moving process cause us a problem, we can stop the balancer process:

 

$mongo #connect to mongos

>use config;

>db.settings.update({_id: "balancer"}, {$set: {stopped: true}}, true)

 

Then mongodb will stop moving chunks, but it will still do the chunk splitting job, and it leaves to us to decide when to move the chunks manually:

 

$mongo #connect to mongos

>use admin;

>db.printShardingStatus();

>db.runCommand({moveChunk: "testdb.users", find: {login: "rock26944"}, to: "shard0000"})

 

We can also write a crond job to do this.

 

Failover

When one shard is down, the query hit this shard will fail, so we will make every shard a replica set. About replica set please refer other documentations.

When one shard(it's a replica set) is down, and the query hit this shard, and the other shard, the query will fail default, but we can add a query option({partial: 1}) to make the query just return partial data, not raise an socket error, currently the ruby mongo driver support this feature, but it's not able to put this config option to a config file, so it's difficult to use this feature right now. But i think the replica set will give us a lot of insurance.

 

Problems

  1. How much memory does mongos need? Will it become a bottleneck?
  2. Does mongos support :read_secondary internal?
  3. How much volumes does the monitor system support in design? We need to estimate the data growth speed.

 

Odds

In sharding environment, we can not use group query, use map/reduce to substitute.

All index must be ascending, can not be descending.

Once you do shard on a collection, it is very difficult to un-shard it (you still can, but tough).

 

Scripts and Step-by-step

I have write a script for staring mongodb sharding environment easily, put this script to your ~/.profile, 

mongod_sharding_restart() {
  killall mongod
  killall mongos
  sleep 2

  [ ! -d /data/configdb ] && mkdir -p /data/configdb
  [ ! -d /data/db ] && mkdir -p /data/db
  [ ! -d /data/shard2_db ] && mkdir -p /data/shard2_db
  rm -f /data/configdb/mongod.lock
  rm -f /data/db/mongod.lock
  rm -f /data/shard2_db/mongod.lock
  #config server
  mongod --fork --logpath /data/shard_config.log --logappend --configsvr &
  #wait for config server
  sleep 3

  #mongos
  mongos --fork --logpath /data/shard_mongos.log --logappend --configdb localhost:27019 &

  #shard1 (mongod)
  mongod --fork --logpath /data/shard_shard1.log --logappend --shardsvr &
  #shard2 (mongod)
  mkdir -p /data/shard2_db
  mongod --fork --logpath /data/shard_shard2.log --logappend --shardsvr --port 37018 --dbpath /data/shard2_db &

  sleep 2
  ps aux | egrep "mongod|mongos"
}
export -f mongod_sharding_restart

and then:

#source ~/.profile

 

Then you can start mongos, config server, and two shards using one command:

#mongod_sharding_restart 

 

 

2.

Connect to mongos:

#mongo

>db.printShardingStatus();

>db.runCommand({addShard: "localhost:27018"});

>db.runCommand({addShard: "localhost:37018"});

>db.printShardingStatus();

  ...

  shards:

      { "_id" : "shard0000", "host" : "localhost:27018" }

      { "_id" : "shard0001", "host" : "localhost:37018" }

  ...

  ...

>db.runCommand({enableSharding: "testdb"});

>db.runCommand({shardCollection: "testdb.collection", key: {_id: 1}})

>db.printShardingStatus();

 

You can check the mongos and shard logs to see the status.

 

>use testdb;

>db.printCollectionStats();