As the US government uses the Internet to monitor and track the news of the public surface, this makes the prospect of the Internet to fulfill its original mission seem bleak. Modern computing technology is helping companies and governments accurately and quickly analyze huge data resources. Among them, the three major technologies of database systems, machine learning and Hadoop infrastructure have played a very important role.
A graph of the average monthly network, email, and data traffic used by consumers worldwide
Only five years ago, it was impossible for government agencies such as the National Security Agency (NSA) to efficiently analyze millions of phone calls, text messages, and online chat records through keywords. Completed task. However, at present, the use of a series of new technologies allows NSA to have relatively sufficient human and financial resources to achieve this. Especially considering that these keywords may eventually prevent the release of future terrorist attacks against the United States.
These new technologies can store a large number of different types of data in a single database, and can achieve high-speed data processing without using expensive hardware equipment, and also do not require data analysis experts to set assumptions in advance.
Tom Davenport, visiting professor and data analysis expert at Harvard Business School, pointed out: "These new technologies have saved huge expenditures for government departments and have greatly improved the ability of government departments to analyze such data. The supporting data center can complete the data analysis task, but the cost of these technologies is much lower than it was a few years ago. "
NSA spent $ 1.2 billion to build a large data center in Utah that will be put into use this fall. It is unclear what computing technology NSA uses in data centers across the United States. But overall, these technologies are divided into three major types:
Database system
Most traditional databases that use the SQL programming language store data in tables composed of rows and columns. However, when it comes to storing character strings including emails or text messages, traditional databases expose the drawback of limited capabilities. And they can't handle pictures or videos.
The new database NoSQL (Not Only SQL, not just SQL), which appeared at the end of 2009, broke through the limitations of traditional databases and allowed data analysis experts to create information requirements for all types of data. These new databases include MongoDB, Cassandra and Simple DB.
The NoSQL database demonstrates extraordinary capabilities in helping companies analyze very large data sets. For example, analysts at US insurance data provider Verisk AnalyTIcs Inc. are constantly running various data models and analysis methods on billions of customer profiles to discover false insurance claim records.
Perry Rotella, vice president and chief information officer of Verisk, said that using the traditional DB2 database provided by IBM "needs to work 6 hours overnight to complete the work." Since then, analysts have to invest a lot of time to study the data results and propose new information requirements, and I am afraid that they will have to stay up for another night. He pointed out that it takes analysts several weeks each time to create a new data model. Verisk has just started using the replaced NoSQL database recently, and analysts can complete the same type of operation in just 30 seconds.
Rotella said: "Suddenly, your model construction bid farewell to the tradition that it only changed once in a few days, and became a real-time update state. By using NoSQL database, you can run data multiple times a day, which is extremely The earth shortens the time to get data results. This feature is simply too powerful. "
For the American online picture trading platform service provider Shutterstock Inc., without NoSQL, the company could hardly survive. Shutterstock has a repository of more than 24 million pictures, and it is increasing at a rate of 10,000 pictures every day. Each of these images has relevant data to help users narrow their search. Shutterstock ’s database also records all the user ’s online behaviors on the site. These behaviors include not only major decisions such as which pictures they authorize, but also minute details, such as where their mouse arrows often stay and where they stay. Specific duration.
MTP And MPO Cable Assembly,Optical Patch Cable,Fibre Optic Cable Assembly,Optical Jumper
Huizhou Fibercan Industrial Co.Ltd , https://www.fibercan-network.com