mongodb-sharding-and-scalability - mongodb-nosql Interview Questions | Preplance

Mobile App Now Available

Master Interviews
Anywhere, Anytime

Get the Preplance app for a seamless learning experience. Practice offline, get daily streaks, and stay ahead with real-time interview updates.

Get it on

Google Play

4.9/5 Rating on Store

2. What are chunks in MongoDB sharding and how does chunk splitting and migration work?

Difficulty: HardType: SubjectiveTopic: Sharding

Chunks are logical groupings of documents based on shard key ranges. MongoDB divides the shard key space into chunks, and each chunk contains documents whose shard key values fall within a specific range. For example, one chunk might contain all documents with user IDs from 1 to 1000, while another contains 1001 to 2000. By default, each chunk has a maximum size of 64 MB. When a chunk grows beyond this size due to inserts or updates, MongoDB automatically splits it into two smaller chunks. Chunk splitting is a metadata operation that updates the config servers; no data is moved during a split. The split point is chosen to divide the chunk into roughly equal sizes. After chunks are split, the balancer monitors the distribution of chunks across shards. If one shard has significantly more chunks than another, the balancer migrates chunks from the heavily loaded shard to less loaded shards. Chunk migration involves copying documents from the source shard to the destination shard, then updating metadata on config servers to reflect the new location. During migration, the chunk being migrated remains on the source shard and continues serving queries until migration completes. Once all documents are copied and verified, the metadata is updated atomically. This ensures that chunk migrations are transparent to applications, though they do consume resources like network bandwidth and disk I/O. The balancer can be configured to run only during specific time windows to avoid impacting production workloads. You can also manually split chunks or move them if needed for special circumstances. However, automatic chunk management works well for most deployments. Chunk size affects migration frequency and efficiency. Larger chunks mean fewer but longer migrations. Smaller chunks mean more frequent but faster migrations. The default 64 MB is a good balance for most workloads.

Example code

// View chunks for a collection
use config
db.chunks.find({ ns: "mydb.users" }).pretty()

// Example chunk
{
  _id: "mydb.users-userId_1000",
  ns: "mydb.users",
  min: { userId: 1000 },
  max: { userId: 2000 },
  shard: "shard0001"
}

// Chunk lifecycle:
// 1. Chunk grows beyond 64MB
// 2. MongoDB splits chunk at midpoint
//    Chunk A: userId 1000-1500
//    Chunk B: userId 1501-2000
// 3. Balancer detects imbalance
// 4. Balancer migrates Chunk B to another shard

// Manual chunk split (rarely needed)
sh.splitAt("mydb.users", { userId: 5000 })

// Change chunk size
use config
db.settings.updateOne(
  { _id: "chunksize" },
  { $set: { value: 128 } },  // 128MB chunks
  { upsert: true }
)

6. Explain the concept of sharding in MongoDB and when you should implement it.

Difficulty: MediumType: SubjectiveTopic: Sharding

Sharding is MongoDB's method for horizontal scaling by distributing data across multiple servers called shards. Each shard is a separate database that holds a portion of the data, and together all shards form the complete dataset. This allows MongoDB to handle datasets and workloads that exceed what a single server can support. In a sharded cluster, data is partitioned based on a shard key, which is a field or fields that exist in every document. MongoDB divides the shard key values into chunks, and each chunk is assigned to a specific shard. As data grows, MongoDB automatically splits chunks and migrates them across shards to maintain balance. You should implement sharding when your dataset exceeds the storage capacity of a single server, typically when approaching several hundred gigabytes or terabytes. Sharding is also appropriate when your read or write throughput exceeds what a single server or replica set can handle, even with adequate hardware. However, sharding adds complexity to your deployment. You need config servers to store cluster metadata, mongos routers to direct queries, and multiple shard replica sets. Only implement sharding when you have exhausted other optimization options like indexing, vertical scaling, and replica set read distribution. Before sharding, ensure your application is ready. Choose your shard key carefully because it cannot be changed after sharding without recreating the collection. A good shard key provides high cardinality, even distribution, and supports your most common query patterns.

Example code

// When to shard:
// 1. Data size > single server storage (500GB-1TB+)
// 2. Working set > available RAM
// 3. Write throughput > single server capacity
// 4. Need to distribute data geographically

// Sharded cluster components:
// Config Servers: store metadata (replica set)
// Mongos Routers: query routing (multiple instances)
// Shards: store data (each is replica set)

// Basic sharding setup
sh.enableSharding("myDatabase")
sh.shardCollection("myDatabase.users", { userId: 1 })

7. Explain the difference between targeted queries and broadcast queries in a sharded cluster. How does this affect performance?

Difficulty: MediumType: SubjectiveTopic: Sharding

In a sharded cluster, queries fall into two categories based on whether they include the shard key: targeted queries and broadcast queries. The difference significantly impacts performance. Targeted queries include the shard key in the query filter. When mongos receives a targeted query, it can determine exactly which shard or shards contain the relevant data by checking the config servers. Mongos then routes the query only to those specific shards. For example, if you query for a specific user ID and user ID is your shard key, mongos routes to only the shard containing that user ID range. This is very efficient because only one shard needs to process the query. Broadcast queries do not include the shard key in the filter. Without the shard key, mongos cannot determine which shards contain relevant data, so it must broadcast the query to all shards. Each shard processes the query independently and returns results to mongos, which then merges the results before returning them to the application. This is much slower because all shards must be queried, network traffic multiplies, and mongos must merge potentially large result sets. The performance impact is substantial. Targeted queries scale linearly; adding more shards does not slow them down because each query still hits only specific shards. Broadcast queries get slower as you add shards because more shards must be queried. In a ten-shard cluster, a broadcast query does ten times the work of querying a single shard. To optimize performance, design your shard key and queries so that common operations are targeted. Include the shard key in query filters whenever possible. For queries that cannot include the shard key, consider using covered indexes on each shard to make the broadcast queries more efficient. Monitor your query patterns using profiling and explain to identify broadcast queries that can be optimized.

Example code

// Shard key: { userId: 1 }

// Targeted query - includes shard key
db.orders.find({ userId: 12345, status: "completed" })
// Mongos knows userId 12345 is on Shard 2
// Routes to Shard 2 only - FAST

// Broadcast query - no shard key
db.orders.find({ status: "completed" })
// Mongos doesn't know which shards have completed orders
// Queries all shards, merges results - SLOW

// Explain shows broadcast
db.orders.find({ status: "completed" }).explain()
// Shows: SHARD_MERGE stage (broadcast to all shards)

// Best practice: include shard key
db.orders.find({
  userId: { $in: [123, 456, 789] },  // Shard key
  status: "completed"
})
// Targeted to specific shards - FAST

8. What are the limitations and challenges of sharding in MongoDB?

Difficulty: MediumType: SubjectiveTopic: Sharding

Sharding provides powerful horizontal scaling but comes with significant limitations and challenges that you must consider before implementation. First, operational complexity increases dramatically. You must deploy and manage config servers, mongos routers, and multiple shard replica sets. This is much more complex than managing a single replica set. Monitoring, backup, and maintenance procedures become more complicated. Second, the shard key is immutable after sharding. Once you shard a collection with a specific shard key, you cannot change it without recreating the collection and migrating all data. Choosing the wrong shard key can cripple performance, and fixing it requires significant downtime and effort. This makes shard key selection a critical decision that must be made carefully. Third, some operations are limited or inefficient in sharded clusters. Unique indexes can only be created on the shard key or fields that include the shard key. This restricts your ability to enforce uniqueness on other fields. Transactions across shards are possible but have performance implications. Aggregation pipelines may require merging results from multiple shards. Fourth, scatter-gather queries that hit all shards are significantly slower than targeted queries. If your query patterns do not include the shard key, performance may actually be worse than an unsharded deployment. This makes query pattern analysis critical before sharding. Fifth, balancing operations consume resources. Chunk migrations use network bandwidth, disk I/O, and CPU. During heavy migration periods, cluster performance can degrade. You can schedule balancing windows, but this adds management overhead. Finally, sharding requires more hardware and infrastructure, increasing costs. You need at minimum three config servers, at least two mongos routers, and multiple shard replica sets. For small datasets, these costs outweigh the benefits.

Example code

// Shard key limitations

// Cannot change shard key after sharding
sh.shardCollection("db.users", { email: 1 })
// Later realize email is poor choice
// Must drop collection and reshhard - significant downtime

// Unique indexes require shard key
// Shard key: { userId: 1 }
db.users.createIndex({ email: 1 }, { unique: true })
// ERROR: cannot create unique index on field without shard key

// Must include shard key
db.users.createIndex({ userId: 1, email: 1 }, { unique: true })
// OK: includes shard key

// Scatter-gather queries are slow
db.users.find({ age: { $gt: 25 } })  // No shard key
// Queries all shards, slow with many shards

// Before sharding: consider alternatives
// - Better indexing
// - Vertical scaling
// - Read replicas for read distribution
// - Application-level caching

11. What factors should you consider when choosing a shard key? Explain the consequences of a poor shard key choice.

Difficulty: HardType: SubjectiveTopic: Shard Key

Choosing a shard key is one of the most critical decisions in sharding because it cannot be changed after sharding without rebuilding the collection. A good shard key must satisfy three main criteria: high cardinality, even distribution, and query pattern alignment. High cardinality means the shard key has many distinct values. This allows MongoDB to distribute data across many chunks and shards. Low cardinality keys, like a status field with only three values, can only create three chunks maximum, preventing effective distribution across many shards. Even distribution means queries and inserts are spread across all shards, avoiding hotspots. Monotonically increasing keys like timestamps or auto-incrementing IDs are poor choices because all new writes go to the highest chunk on one shard. This creates a write hotspot that defeats the purpose of sharding. Query pattern alignment means your most common queries should include the shard key. When queries include the shard key, mongos can route them directly to specific shards. Without the shard key in queries, mongos must broadcast to all shards, which is inefficient and slow. Poor shard key choices lead to several problems. Uneven distribution causes some shards to fill up while others remain empty, wasting resources and limiting scalability. Hotspots concentrate all activity on one shard, creating bottlenecks. Scatter-gather queries that hit all shards are slow and resource-intensive. A common strategy is using a compound shard key that combines a field with good distribution, like user ID, with a time-based field for efficient time-range queries. Another approach is using hashed shard keys for monotonically increasing values to ensure even distribution.

Example code

// Good shard key: compound with high cardinality
sh.shardCollection("db.orders", { userId: 1, orderDate: 1 })
// Pros: high cardinality, supports user queries, time-based queries
// Query: db.orders.find({ userId: 123, orderDate: {...} })
// Routes to specific shard

// Poor shard key: low cardinality
sh.shardCollection("db.orders", { status: 1 })
// Cons: only 3-4 values (pending, completed, cancelled)
// Cannot distribute beyond 3-4 chunks

// Poor shard key: monotonically increasing
sh.shardCollection("db.orders", { _id: 1 })
// Cons: all new writes to one shard (hotspot)
// Better: { _id: "hashed" }

// Queries without shard key (slow)
db.orders.find({ productId: 456 })
// Mongos broadcasts to all shards

12. Describe the components of a MongoDB sharded cluster and how they work together.

Difficulty: HardType: SubjectiveTopic: Sharding

A MongoDB sharded cluster consists of three main components: shards, config servers, and mongos routers. Each component plays a specific role in distributing data and routing queries. Shards are the actual data stores in a sharded cluster. Each shard is typically deployed as a replica set to provide high availability and data redundancy. Shards hold subsets of the data based on the shard key ranges assigned to them. For example, one shard might hold documents with user IDs from 1 to 10000, while another holds 10001 to 20000. Config servers store metadata about the cluster configuration. This includes which shards exist, what chunks of data each shard contains, and the ranges of shard key values in each chunk. Config servers must be deployed as a replica set for high availability because they are critical to cluster operations. If config servers are unavailable, the cluster cannot route queries or perform administrative operations, though existing connections continue to work. Mongos routers are the query routing layer that applications connect to. Applications never connect directly to shards. Instead, they connect to mongos instances, which appear as normal MongoDB servers. When mongos receives a query, it consults the config servers to determine which shards contain relevant data. For targeted queries that include the shard key, mongos routes to specific shards. For scatter-gather queries without the shard key, mongos broadcasts to all shards and merges results. The balancer is a background process that monitors chunk distribution across shards. When it detects imbalance, it migrates chunks from heavily loaded shards to less loaded ones. This ensures even distribution as data grows. Together, these components enable transparent horizontal scaling. Applications interact with mongos routers as if querying a single database, while data is distributed across multiple shards for scalability and performance.

Example code

// Sharded cluster architecture:
//
// Application
//     ↓
// Mongos (Query Router) ← queries config servers for metadata
//     ↓
// Config Servers (Metadata) - replica set
//     ↓
// Shards (Data Storage) - each is replica set
//   Shard 1: userId 1-10000
//   Shard 2: userId 10001-20000
//   Shard 3: userId 20001-30000

// Query flow:
// 1. App sends: db.users.find({ userId: 15000 })
// 2. Mongos checks config: "userId 15000 is on Shard 2"
// 3. Mongos routes query to Shard 2 only
// 4. Shard 2 returns results
// 5. Mongos returns to application

13. Explain the difference between horizontal and vertical scaling. Why does MongoDB prefer horizontal scaling?

Difficulty: MediumType: SubjectiveTopic: Scalability

Vertical scaling means adding more resources to a single server, such as upgrading CPU, adding more RAM, or using faster storage. It is the simpler approach because your architecture remains unchanged; you just use more powerful hardware. However, vertical scaling has hard limits. There is a maximum amount of RAM, CPU, and storage you can add to a single machine, and costs increase exponentially at the high end. Horizontal scaling means adding more servers to distribute the load across multiple machines. Instead of one powerful server, you use many commodity servers working together. MongoDB implements horizontal scaling through sharding, where data is partitioned across multiple shards. MongoDB prefers horizontal scaling for several reasons. First, it has virtually unlimited capacity. You can add shards indefinitely as your data and workload grow. There is no practical upper limit like there is with vertical scaling. Second, it is more cost-effective. Adding commodity servers is cheaper than buying enterprise-grade high-end hardware. Third, horizontal scaling provides better fault tolerance. With data distributed across multiple servers, the failure of one shard does not take down the entire system. When combined with replica sets on each shard, you have both high availability and horizontal scalability. Fourth, it enables geographic distribution. You can place shards in different data centers or regions to reduce latency for users in different locations. However, horizontal scaling adds complexity. You need to manage multiple servers, choose good shard keys, and handle distributed queries. Vertical scaling remains viable for smaller deployments or when your data fits on a powerful single server. Many deployments use both strategies: vertical scaling for individual servers and horizontal scaling across multiple shards.

Example code

// Vertical scaling (single server)
// Year 1: 16GB RAM, 4 cores, 500GB storage
// Year 2: 64GB RAM, 16 cores, 2TB storage
// Year 3: 256GB RAM, 32 cores, 8TB storage
// Eventually: Cannot scale further, very expensive

// Horizontal scaling (sharding)
// Year 1: 3 shards, 16GB RAM each
// Year 2: 6 shards, 16GB RAM each
// Year 3: 12 shards, 16GB RAM each
// Can continue adding shards indefinitely

// Sharding setup for horizontal scaling
sh.addShard("shard1/host1:27017")
sh.addShard("shard2/host2:27017")
sh.addShard("shard3/host3:27017")
// Add more shards as needed

14. What is zone sharding in MongoDB and when would you use it?

Difficulty: HardType: SubjectiveTopic: Sharding

Zone sharding, also called tag-aware sharding, allows you to control which shards store specific ranges of data based on shard key values. You create zones, associate shards with zones, and define shard key ranges for each zone. The balancer then ensures chunks falling within zone ranges are migrated to shards associated with those zones. Zone sharding is useful for several scenarios. First, geographic data distribution where you want to keep data close to users. For example, you can create US zone and EU zone, assign shards in US data centers to US zone and shards in EU data centers to EU zone, then define ranges like user IDs starting with 1 for US and 2 for EU. Second, data tiering based on access patterns. You might have hot data that is frequently accessed and cold data that is rarely accessed. Create hot zone on fast SSD-backed shards and cold zone on cheaper HDD-backed shards, then assign recent data to hot zone and old data to cold zone based on date ranges. Third, multi-tenancy where you want to isolate different customers on different hardware. Create zones for different customer tiers, like premium zone on powerful hardware and standard zone on regular hardware, then assign customers to appropriate zones. To implement zone sharding, first add shards to zones using sh.addShardToZone. Then define the ranges of shard key values for each zone using sh.updateZoneKeyRange. The balancer automatically migrates chunks to respect zone boundaries. This happens gradually and transparently. Zone sharding adds complexity because you must manage zone definitions and shard assignments. Use it only when you have clear requirements for data placement. For most applications, automatic chunk distribution without zones is sufficient.

Example code

// Geographic zone sharding

// 1. Add shards to zones
sh.addShardToZone("shard-us-west", "US")
sh.addShardToZone("shard-us-east", "US")
sh.addShardToZone("shard-eu-west", "EU")

// 2. Define shard key ranges for zones
sh.updateZoneKeyRange(
  "mydb.users",
  { userId: 1000000, country: "US" },
  { userId: 1999999, country: "US" },
  "US"
)

sh.updateZoneKeyRange(
  "mydb.users",
  { userId: 2000000, country: "EU" },
  { userId: 2999999, country: "EU" },
  "EU"
)

// 3. Balancer migrates chunks to appropriate shards
// US users' data stays on US shards
// EU users' data stays on EU shards

// Result: reduced latency for users
// US user queries hit US shards (low latency)
// EU user queries hit EU shards (low latency)

Master Interviews
Anywhere, Anytime

MongoDB Sharding & Scalability – MongoDB/NoSQL interview questions

Set overview

MongoDB/NoSQL – MongoDB Sharding & Scalability questions (14)

1. Which characteristic is MOST important when choosing a shard key?

Continue your preparation

2. What are chunks in MongoDB sharding and how does chunk splitting and migration work?

3. What do config servers store in a MongoDB sharded cluster?

4. What is sharding in MongoDB?

5. What is the role of mongos in a sharded cluster?

6. Explain the concept of sharding in MongoDB and when you should implement it.

7. Explain the difference between targeted queries and broadcast queries in a sharded cluster. How does this affect performance?

8. What are the limitations and challenges of sharding in MongoDB?

9. What is the main advantage of hashed sharding over range-based sharding?

10. What triggers chunk migration in a sharded cluster?

11. What factors should you consider when choosing a shard key? Explain the consequences of a poor shard key choice.

12. Describe the components of a MongoDB sharded cluster and how they work together.

13. Explain the difference between horizontal and vertical scaling. Why does MongoDB prefer horizontal scaling?

14. What is zone sharding in MongoDB and when would you use it?

More sets in MongoDB/NoSQL

Explore related subjects

Master Interviews Anywhere, Anytime

MongoDB Sharding & Scalability – MongoDB/NoSQL interview questions

Set overview

1. Which characteristic is MOST important when choosing a shard key?

Continue your preparation

2. What are chunks in MongoDB sharding and how does chunk splitting and migration work?

3. What do config servers store in a MongoDB sharded cluster?

4. What is sharding in MongoDB?

5. What is the role of mongos in a sharded cluster?

6. Explain the concept of sharding in MongoDB and when you should implement it.

7. Explain the difference between targeted queries and broadcast queries in a sharded cluster. How does this affect performance?

8. What are the limitations and challenges of sharding in MongoDB?

9. What is the main advantage of hashed sharding over range-based sharding?

10. What triggers chunk migration in a sharded cluster?

11. What factors should you consider when choosing a shard key? Explain the consequences of a poor shard key choice.

12. Describe the components of a MongoDB sharded cluster and how they work together.

13. Explain the difference between horizontal and vertical scaling. Why does MongoDB prefer horizontal scaling?

14. What is zone sharding in MongoDB and when would you use it?

More sets in MongoDB/NoSQL

Explore related subjects

Master Interviews
Anywhere, Anytime