With rapid development of the Internet era, enterprise data is increasing explosively. An urgent demand is developed on how to generate value from the complex and chaotic data and how to use the data to implement innovations and seize business opportunities. Obviously, conventional data processing is incompetent in this big data era.
MapReduce Service (MRS) provides storage and analysis capabilities for massive data and builds a reliable, secure, and easy-to-use operation and maintenance (O&M) platform. Users can apply for use Hadoop, Spark, HBase, and Hive services to quickly create clusters and provide storage and computing capabilities for massive data analysis or real-time processing. After data storage and computing are fulfilled, the cluster service can be terminated, and no fee will be charged accordingly. You can also choose to run clusters permanently.
MRS delivers the following functions:
- Analysis and computing of massive data
Hadoop 2.7.2: Based on Hadoop applying a distributed system infrastructure, MRS uses MapReduce to implement parallel computing of large data sets (TB-level or above).
Spark 1.5.1: Spark is a distributed batch processing framework. It provides analysis and mining and iterative memory computing capabilities and supports application development in multiple programming languages, including Scala, Java, and Python. Additionally, it provides Spark SQL, which enables data to be queried and analyzed using structured query language (SQL) statements.
HBase 1.0.2: Hadoop Database (HBase) is a column-based distributed storage system that features high reliability, performance, and scalability. HBase is designed to supplement relational databases in processing massive data.
Hive 1.3.0: Hive is a data warehouse framework built on Hadoop. It stores structured data using the Hive query language (HQL), a language like the SQL. Hive converts HQL statements to MapReduce or HDFS tasks for querying and analyzing massive data stored in Hadoop clusters.
Hadoop Distributed File System (HDFS) features high fault tolerance and provides high-throughput data access, applicable to the processing of large data sets. After being processed and analyzed, data is encrypted by using Secure Sockets Layer (SSL) and transmitted to the Object Storage Service (OBS) system or HDFS.
A highly available commercial Hadoop big data platform can be built by performing a few steps within a few minutes.
MRS provides a user-friendly web-based console, enabling you to perform management operations with ease.
99.9% service availability: Critical services of MRSuch as NameNode and HMaster, are working in active-standby mode. In the event of an active server failure, services are automatically switched over to the standby server within minutes.
- MRS supports multiple storage types
(1) OBS: Encrypted transmission: Data is transmitted to OBS using SSL and table-level encryption is supported, ensuring data security.
(2) Elastic Volume Service (EVS): For the preceding storage types, data is stored in three copies. Additionally, Hadoop employs the three copies mechanism. For this reason, user data is stored in nine copies (3 x 3), remarkably improving data reliability.
(3) Local storage (Only d1.4xlarge and d1.8xlarge ECS instance support.)