IT - big data

• The term Big Data has been used to refer to a range of problems and technologies that relate to the management of very large data sets. The amount of data that we currently generate is enormous, and is increasing each year.

  “IBM say that ‘every day, we create 2.5 quintillion bytes of data – so much that 90 per cent of the data in the world today has been created in the last two years alone.’”

  Sources of Big Data

  Social media platforms produce huge quantities of data, both from individual network profiles and the content that influencers and the less influential alike produce (e.g. you-tubers, everyday technology users). Short form blogging, link-sharing, expert blog comments, user forums, ‘likes’ and more all contain potentially useful information that can be mined.

  There is also data produced through sheer activity, for example machine-generated
• What is characterized as the 'internet of things'?

  Interconnection of devices via the Internet
• What does software-as-a-service provide?

  Online access to software via subscription
• What type of data can be found in legacy systems?

  Medical records
• What are some sources of data mentioned?

  • Text, images, and videos on social media
  • Medical records
  • Fitness monitors (wearable technology)
  • Financial markets data
  • Web server logs
  • Mobile phone records
  • E-commerce transactions
  • Device logs (internet of things)
• What is a common definition issue with big data?

  There is no generally agreed definition
• What are the three Vs of big data problems?

  Volume, velocity, and variety
• Why is it difficult to define the size limit of big data?

  What was big yesterday may not be today
• What traditional tools struggle with big data problems?

  Relational database management systems
• What does velocity in big data refer to?

  Real-time flows of data
• How many transactions per day were estimated on the London Stock Exchange in September 2016?

  891 thousand transactions
• What is the significance of CCTV cameras in the UK regarding data?

  They generate a large amount of data
• What is a challenge for the big data industry?

  Managing volume, velocity, and variety of data
• What type of data is often unstructured in big data?

  Video, audio, and image data
• What is a characteristic of big data problems?

  They involve large and complex data sets
• What type of databases were used to handle large data sets in the past?

  Relational databases and data warehouses
• Why are relational databases and data warehouses insufficient for current data volumes?

  They can't handle the volumes of data produced
• What is Hadoop?

  A collection of open-source software utilities
• What is the role of data analytics in big data management?

  To interpret data and extract useful information
• What do algorithms do in data analytics?

  Spot patterns, relationships, and trends
• What techniques may be included in data analytics algorithms?

  Statistical analysis and artificial intelligence
• What are potential applications of big data and analytics?

  • Healthcare
  • Crime prevention
  • Energy management
• How do databases interact with social networking sites?

  They store personal details entered by users
• What is a database?

  A structured collection of data items
• What is the most common structure of a database?

  A collection of interrelated tables
• What is a database table?

  A group of similar data with rows and columns
• What is a record in a database?

  A basic unit of data stored in a data file
• What is a field in a database?

  A part of a record for a single data item
• What are keys in a database?

  Fields that identify records or establish relationships
• What is a primary key?

  A field that uniquely identifies a record
• What is a foreign key?

  A primary key from one table in another table
• How are relationships represented in a relational database?

  Using primary and foreign keys
• What types of relationships can exist in a relational database?

  One-to-many, many-to-many, one-to-one
• What is the purpose of a query in a database?

  To search and extract data from a database