The Package for Memcached Cluster integrates six DragonflyDB instances and three McRouter nodes, delivering a cutting-edge caching solution optimized for modern applications. Designed for maximum scalability and performance, this package allows developers to deploy distributed caching layers seamlessly, reduce database strain, and enhance response times significantly.
Why Choose DragonflyDB Cluster?
- Simplified Scalability: Deploy a highly efficient distributed caching layer with pre-configured Docker packages, reducing complexity and ensuring faster response times.
- Intelligent Request Routing: Utilize McRouter’s advanced features, such as prefix routing, replicated pools, and failover mechanisms, for seamless cache operation.
- Blazing-Fast Performance: Built on DragonflyDB, delivers a high-speed, fault-tolerant caching system optimized for high-demand environments.
- Dynamic Configuration: Manage and scale your caching infrastructure with live updates, ensuring zero downtime.
- Multi-Level Caching: Implement tiered caching for enhanced data retrieval efficiency.
Core Features
- High Availability: Ensure reliable data storage with integrated failover and replication mechanisms.
- Optimized Caching: Route requests efficiently across multiple Memcached servers using McRouter for better load balancing and resource management.
- Advanced Routing Logic: Configure routing policies based on prefixes, clusters, or custom requirements using McRouter’s modular routing handles.
- Cold Cache Warmup: Seamlessly bring new cache nodes online without impacting application performance.
- Rich Monitoring Tools: Access detailed statistics and debugging tools for complete visibility into cache performance.
- Security First: Protect data in transit with SSL support and IPv6 compatibility.
- Multi-Threaded Architecture: Leverage multi-core systems for efficient request handling.
Easy Deployment
This package includes pre-configured Docker containers and a straightforward setup process, enabling developers to quickly build a scalable, reliable caching infrastructure.
Included in Your Offer
- One Reverse proxy: With load balancer optimized for high traffic
- Three Nginx Server: Optimized for web traffic management.
- One o more PHP Server: Pre-configured for Memcached integration.
- Three McRouter Instances: Intelligent request routing for optimal caching.
- Six DragonFlyDB Memcached Instances: Integrated with your existing database and website.
Note: Each “server” refers to one Docker container. All containers can run on the same physical machine or in the cloud for performance and redundancy.
Recommended Server Specifications
For the Cluster configuration, we recommend using a cloud server instance with the following specifications:
- Minimum RAM: 24 GB (This is the minimum requirement to run the cluster effectively for production.) We have tested the application with 12 GB of RAM, and it functions perfectly, as Docker consumes very little memory. However, to achieve maximum speed, the application requires this amount of RAM, which also depends on the quantity of cache on your server. If the RAM fills up quickly, the application’s performance will degrade rapidly, as it runs solely in RAM.
- The data is saved both on disk, and a snapshot of the program data is created. If your server has less memory, only a single instance of DragonflyDB can be run. However, in this case, the cluster loses its significance. A single instance of DragonflyDB requires a minimum of 4 GB of RAM to run effectively.
- Maximum RAM Supported: Each DragonflyDB instance can support up to 768 GB of RAM, providing extensive caching capabilities. In a configuration with 6 instances, comprising 3 master instances and 3 slave instances, the total RAM available is 4608 GB. However, due to the mirrored setup (replication), the effective storage capacity is halved, resulting in 2304 GB of usable RAM. This configuration is highly cost-effective and suitable for applications with moderate to high traffic.
- The DragonflyDB Cluster project is paving the way for a new generation of caching systems, delivering exceptional performance and scalability. Designed to handle demanding workloads, this advanced configuration makes it an ideal choice for large-scale applications.
- The application can set a limit on the RAM that can be used. If you have unlimited memory, you can set it to manage automatically, and DragonflyDB will manage its memory based on the available memory on the server.
However, a custom cluster can be configured without limitations and can be horizontally scaled across different geographical areas or within the same data center based on your preferences. For more advanced options, this custom cluster can run in a Multi-Cloud, Multi-Cluster Kubernetes Orchestration setup with Karmada across multiple data centers and with various cloud providers globally, without any limitations.
Handle with Care: Development Status
As a project still under development, the DragonflyDB Cluster team advises users to:
- Test thoroughly in controlled environments before production deployment.
- Adjust configurations to match individual use cases for optimal results.
- Verify licenses to ensure compliance with all third-party dependencies and integrations.
Licenses
- DragonflyDB Cluster: MIT License
- DragonflyDB: DragonflyDB License
- McRouter: McRouter License
- Memcached: Memcached License
Important Notice Regarding DragonflyDB Usage
In compliance with the Dragonfly Business Source License 1.1 (BSL 1.1):
- Permitted Use: This project involves installing and configuring DragonflyDB as part of your private caching infrastructure.
- Prohibited Use: You cannot offer DragonflyDB as a hosted or managed service, nor provide any solution that allows third parties (other than your employees or contractors) to access or use DragonflyDB features.
- One-Time Installation: You can only order a one-time installation for a managed solution. You need to contact the DragonflyDB team to ensure that the service fully respects the licensing terms of DragonflyDB, or you can manage it yourself.
Explore and Contribute
Learn more and become part of the journey by visiting the DragonflyDB Cluster repository on GitHub
Multiple Configurations for Diverse Needs
DragonflyDB Cluster provides several configurations to suit a variety of use cases:
- Basic setup: for smaller-scale applications or testing environments.
- Advanced setup: featuring three routers, three master nodes, and three slave nodes, optimized for performance, fault tolerance, and high availability. If you have a minimum of three servers available for caching, we can install a package on each of them. Each server will have one router and two instances: one DragonflyDB master and one DragonflyDB slave. Additionally, we will set up three Nginx proxy servers in front, along with a load balancer.
- For high availability (HA), a total of six instances are required, which means you will need a minimum of six servers, each with one instance of DragonflyDB. In this case, you will have three master nodes and three slave nodes operating in a mirrored setup, so the available memory will be effectively halved.
High Availability (HA) Configuration
For full high availability, a minimum of six servers is required, each hosting one DragonflyDB instance or a cloud server. This setup utilizes three pools:
- Two active pools: Each consists of one master and one slave, configured for sharding.
- One could pool (FailoverPool): Remains empty but is ready to take over in case of a failure in one of the active pools.
This ensures uninterrupted services and system reliability, even during unexpected failures.
- Maximum RAM utilization: Up to 1536 GB, making it suitable for applications with moderate to high traffic.
- Automated failover: If a master node fails, its corresponding slave instantly takes over, minimizing downtime.
- Load balancing and redundancy: The mirrored structure ensures efficient data distribution and performance optimization.
This architecture makes DragonflyDB an excellent choice for mission-critical applications requiring both scalability and high availability.
Routing Configuration with Sharding
To manage request distribution while utilizing sharding, the following McRouter configuration is applied. This configuration ensures a robust, scalable, and high-performance caching solution. By leveraging two active pools with sharding and a standby pool for failover, DragonflyDB Cluster guarantees seamless operation and efficient resource utilization, making it the optimal choice for large-scale, high-traffic applications.
Key Advantages
- Unparalleled Speed: With performance that can reach up to 25x faster than traditional Memcached, DragonflyDB Cluster is designed for use cases where speed is critical.
- Scalability: The cluster’s advanced architecture supports scaling up to 4.6 TB, offering exceptional capacity for handling massive datasets.
- Flexibility: Adaptable to a wide range of scenarios, users can customize configurations to optimize for their specific requirements.
Preconfigured packages:
For preconfigured basic packages, you can find them on the app store:
- DragonflyDB Cluster - Basic 2x1x1: Starting from 4GB - 768 GB of usable RAM
- DragonflyDB Cluster - Advance 2x2x2: Starting from 8GB - 1536 GB of usable RAM
- DragonflyDB Cluster - High Availability (HA) 3X3X3: Starting from 8GB - 1536 GB of usable RAM with cold storage failover, reverse proxy and load balancer optimized for high traffic
Please note that we do not offer future support for any implementation solution; this offer does not include a managed service if you require a managed solution, please contact the Dragonfly team.
Custom Installation Assistance
If you would like custom installation of this product, please contact me at olariu_romeo@yahoo.it . I will be happy to help you with the installation process and ensure a smooth setup.
- Based on the size and complexity of your project, we will provide you with a tailored pricing quote.
- For purchasing a custom installation service, please visit the following link: DragonflyDB Cluster -Custom
After Purchase Notes
After your purchase, please provide the following information via email:
- Server login credentials
- An SSH key for secure access
- Details about the project you wish to integrate
Additional Documentation
For more details, please refer to the main repository:
Disclaimer
This product is provided "as is," without any guarantees or warranties regarding its functionality, performance, or reliability. By using this product, you acknowledge that you do so at your own risk. Romulus and its contributors are not liable for any issues, damages, or losses that may arise from the use of this product. We recommend thoroughly testing the product in your own environment before deploying it in a production setting.
Configuration Diagram:
+-------+-------+
| CLIENT |
+-------+-------+
|
v
+-------+-------+
| PUBLIC |
| NETWORK |
+-------+-------+
|
v
+-------+-------+
| DNS SERVER |
+-------+-------+
|
v
+-------+-------+
| REVERSE PROXY |
+-------+-------+
|
v
+-------+-------+
| LoadBalancer |
+-------+-------+
|
v
+-------+-------+ +-------+-------+
| PRIVATE | | YOUR UNA |
| NETWORK | | APP |
+-------+-------+ +-------+-------+
| |
v |
+-------+-------+ |
| NGINX | v
+ +<------------->+<-------------------->+<---> OTHER SERVICES
| PROXY CLUSTER | |
+-------+-------+ |
| |
v |
+<--------------------->+<--------------------->+ |
| | | |
v v v v
+---------------+ +------+-------+ +-------+-------+ +-------+-------+
| McRouter 1 | | McRouter 2 | | McRouter 3 | | DB Cluster 1 |
+-------+-------+ +------+-------+ +-------+-------+ +-------+-------+
| | | |
v v v |
+<--------------------->+<--------------------->+ |
| | | |
v v v v
+-------+-------+ +------+-------+ +-------+-------+ +-------+-------+
| LoadBalancer | | LoadBalancer | | FailoverPool | | DB |
| Pool A | | Pool B | | Cold Pool | | LoadBalancer |
+-------+-------+ +------+-------+ +-------+-------+ +-------+-------+
| | | |
v v v |
+<--------------------->+<--------------------->+ |
| | | |
v v v v
+-------+-------+ +------+-------+ +-------+-------+ +-------+-------+
| DragonflyDB | | DragonflyDB | | DragonflyDB | | Mysql |
| master1 | | master2 | | master3 | | master1 |
+-------+-------+ +------+-------+ +-------+-------+ +-------+-------+
| | | |
v v v v
+-------+-------+ +------+-------+ +-------+-------+ +-------+-------+
| DragonflyDB | | DragonflyDB | | DragonflyDB | | Mysql |
| slave1 | | slave2 | | slave3 | | slave1 |
+-------+-------+ +------+-------+ +-------+-------+ +-------+-------+