Assignments Of Day 25: MongoDB Administration (Advanced)
Assignment 1: Enabling Database Profiling in MongoDB
Task:
Enable database profiling for operations that take longer than 100ms and then query the system.profile collection to view the logged operations.
Solution:
1. Enable Profiling: First, open your MongoDB shell and set the profiling level to 1 with a slow query threshold of 100ms:
javascript
CopyEdit
db.setProfilingLevel(1, { slowms: 100 });
2. Perform Some Operations: Run a few operations like inserting, updating, or querying some data. For example:
javascript
CopyEdit
db.users.insert({ name: "John Doe", age: 30 });
db.users.find({ age: 30 });
3. Check Profiling Data: To view the recorded operations, query the system.profile collection:
javascript
CopyEdit
db.system.profile.find().pretty();
This will show operations that took longer than 100ms.
4. Disable Profiling (optional): Once done, you can turn off profiling to avoid performance overhead:
javascript
CopyEdit
db.setProfilingLevel(0);
Assignment 2: Analyzing Slow Queries from system.profile
Task:
Find and analyze slow queries from the system.profile collection.
Solution:
1. Enable Profiling (if not already done):
javascript
CopyEdit
db.setProfilingLevel(1, { slowms: 100 });
2. Perform Some Slow Queries: Run operations that may take more than 100ms. For example:
javascript
CopyEdit
db.users.find({ age: { $gt: 30 } });
3. View and Analyze Slow Queries: Query the system.profile collection to find slow queries:
javascript
CopyEdit
db.system.profile.find({ millis: { $gt: 100 } }).pretty();
4. Interpret the Data: Look at the logged operations, especially the millis field (which shows execution time). Long-running queries could be optimized by adding indexes.
Assignment 3: Creating Indexes to Improve Query Performance
Task:
Create an index to optimize a query from the previous assignment, and then re-run the query to observe the performance improvement.
Solution:
1. Create Index: Assuming that the query is slow because we are querying the age field, create an index on the age field:
javascript
CopyEdit
db.users.createIndex({ age: 1 });
2. Re-run the Slow Query: Run the same query again:
javascript
CopyEdit
db.users.find({ age: { $gt: 30 } });
3. Check Profiling Data: Query the system.profile collection to ensure the query is using the index:
javascript
CopyEdit
db.system.profile.find({ millis: { $gt: 100 } }).pretty();
4. Compare Execution Times: Observe the millis field before and after creating the index. The execution time should now be faster.
Assignment 4: Configuring MongoDB Log Level
Task:
Change the log verbosity level of MongoDB to 2 and analyze the logs for detailed information.
Solution:
1. Change Log Verbosity: To change the log verbosity, use the following command:
bash
CopyEdit
mongod --logpath /path/to/mongo.log --verbosity 2
2. Perform Some Database Operations: Insert, update, and query some documents to generate log entries.
3. Check the Logs: Open the log file located at /path/to/mongo.log and look for detailed information. With verbosity set to 2, the logs will contain more granular details, including query execution information and warnings.
Assignment 5: Analyzing MongoDB Logs for Errors
Task:
Check the MongoDB logs for any error messages related to replication or database connections.
Solution:
1. View MongoDB Logs: Use the tail command to view the latest entries in the MongoDB log:
bash
CopyEdit
tail -f /var/log/mongodb/mongod.log
2. Identify Errors: Look for any error messages, such as "connection refused," "replication lag," or "out of memory." These are often highlighted as error or warning messages.
3. Interpret the Errors:
o If you see errors related to replication, ensure the replica set configuration is correct and that network connections are stable.
o If connection errors appear, check the MongoDB server's availability and firewall settings.
Assignment 6: Configuring Log Rotation
Task:
Configure MongoDB to rotate logs when they reach 100MB in size.
Solution:
1. Modify MongoDB Configuration File (mongod.conf): Open the mongod.conf file and add the following configuration for log rotation:
yaml
CopyEdit
systemLog:
destination: file
path: /var/log/mongodb/mongod.log
logRotate: rename
size: 100MB
2. Restart MongoDB: After making changes, restart the MongoDB service:
bash
CopyEdit
sudo systemctl restart mongod
3. Check Log Rotation: Generate some logs by performing operations and check the log file size. Once it reaches 100MB, MongoDB will rotate the logs and create a new file.
Assignment 7: Finding Replication Issues in MongoDB Logs
Task:
Examine the MongoDB logs for any replication issues such as replication lag.
Solution:
1. Check Logs for Replication Entries: Use the tail command to view the logs and look for replication-related entries:
bash
CopyEdit
tail -f /var/log/mongodb/mongod.log | grep "replication"
2. Look for Replication Lag: If you see messages indicating replication lag or failed syncs, this means that the secondary nodes are not catching up with the primary. This could be due to network issues or high write load on the primary.
3. Resolve Replication Issues:
o Ensure that all nodes are properly connected to the replica set.
o Check network connectivity and disk I/O performance.
o Consider increasing the write concern to ensure data is safely replicated.
Assignment 8: Analyzing the Impact of Sharding on Query Performance
Task:
Examine how sharding affects query performance by comparing query execution times on a sharded MongoDB cluster versus a non-sharded instance.
Solution:
1. Set Up Sharding:
o Ensure you have a MongoDB cluster with sharding enabled.
o Shard a collection, for example, the users collection:
javascript
CopyEdit
sh.shardCollection("mydb.users", { age: 1 });
2. Perform Queries Before and After Sharding:
o Run a query on the users collection before sharding:
javascript
CopyEdit
db.users.find({ age: { $gt: 30 } });
o Then, shard the collection and run the same query again.
3. Compare Execution Times: Use system.profile to compare query execution times before and after sharding. Sharding should improve query performance by distributing the load across multiple nodes.
Assignment 9: Collecting Performance Metrics Using mongostat
Task:
Use mongostat to collect performance metrics from MongoDB and analyze them.
Solution:
1. Run mongostat: Open a terminal and run the following command to monitor MongoDB performance:
bash
CopyEdit
mongostat --host localhost --port 27017
2. Interpret Output: The output will show metrics such as:
o Inserts/sec
o Queries/sec
o Updates/sec
o Network traffic
o Memory usage
Analyze these metrics to understand database performance and identify potential bottlenecks.
Assignment 10: Setting Up and Using the mongotop Tool
Task:
Use the mongotop tool to monitor the read/write operations on your MongoDB instance.
Solution:
1. Run mongotop: Open a terminal and run the following command:
bash
CopyEdit
mongotop --host localhost --port 27017
2. Monitor Read/Write Operations: The mongotop tool will show you the time spent reading and writing to each collection. This helps in identifying which collections are heavily accessed.
3. Interpret the Output: Look for collections with high read/write times. This could indicate areas where optimization or indexing may be needed.
Conclusion
These assignments help students practice and apply advanced MongoDB administrative tasks, such as profiling, log analysis, indexing, and performance monitoring. Each task involves practical steps that are essential for ensuring MongoDB performs optimally in production environments.
