DATABASES + BIG DATA + FUNCTIONAL PROGRAMMING

• Databases


  Made from tables which store multiple entities
• Foreign key


  An attribute in a table which is the primary key in another
• Normalisation


  Allows efficiency without compromising integrity of data stores
• Normalisation


  • Solves duplication and redundancy of data
  • Improves data consistency
  • Reduces update insertion and deletion errors
• Client server database


  Provides simultaneous access for multiple clients e.g. social media websites
• Concurrent access


  Occurs when users attempt to access the same field at the same time
• Techniques to manage concurrent access

  • Record locks
  • Serialisation
  • Timestamp ordering
  • Commitment ordering
• Record locks


  When one record is accessed by a user it is immediately locked to all other users so they cannot access or modify it
• Serialisation


  Rather than locking, requests from other users are placed in a queue to execute sequentially
• Timestamp ordering


  When multiple commands are sent to the same field, each is assigned a timestamp where each command was initiated. They are executed in order of the timestamp order
• Commitment ordering


  Algorithm is used to work out optimum order in which to execute commands to the same field. Algorithm considers impact of commands on other parts of the database to avoid issues
• Big data


  Defined by volume, velocity and variety
• Volume of big data


  • Too much data for it to fit on a single hard drive or server- it has to be stored over multiple servers, each composed of multiple hard drives
• Velocity of big data


  • Data is created and modified rapidly- the servers must respond to frequently changing data within milliseconds
• Variety of big data


  • Data consists of many types from binary files to photos and videos
• Big data's unstructured nature makes it difficult to analyse as conventional databases are suited to conform to a row column structure and do not scale well across multiple servers
• In order to extract useful information from big data machine learning techniques must be used to discern patterns
• Big data


  • Air traffic control
  • Global banking systems
• Functional programs


  Stateless and make use of immutable data structures. The FP paradigm supports higher order functions
• Higher order function


  A function which takes functions as its input and/or outputs a function
• Fact


  Immutable and cannot be overwritten. Each stored with a timestamp so will always be true and computer can find which value is most recent
• Fact based model


  Does not index data instead appending new data to existing data set
• First class object


  Objects which may appear in expressions, be assigned to a variable, be assigned as arguments, be returned as function calls
• Functions are first class objects so can be passed as arguments or returned as the result of another function
• Functional composition


  Combining two functions to create a new one, denoted by a little circle
• Higher order functions


  • Either take a function as an argument and/or return a function as a result
• Map


  Does the function to every item in a list
• Filter


  Returns elements of the list that adhere to the condition given- it filters in rather than out
• Fold


  Reduces the list to a single value by doing a subtracting or adding every element in a list to a value