BRK2555: NoSQL on Microsoft Azure: An introduction

[Speaker: David Chappell] NoSQL data stores have gone mainstream. In this session, David Chappell will survey the expanding NoSQL landscape on Azure, providing a big-picture view of these technologies. He’ll also spend a bit more time on DocumentDB, Microsoft’s document database-as-a-service, and Azure Search. This session is aimed at anybody who’s new to NoSQL or is curious about the NoSQL options that Azure provides today.

How toBreakoutIgnite2015
1.0x

BRK2555: NoSQL on Microsoft Azure: An introduction

Created 2 years ago

Duration 1:15:46
lesson view count 24
[Speaker: David Chappell] NoSQL data stores have gone mainstream. In this session, David Chappell will survey the expanding NoSQL landscape on Azure, providing a big-picture view of these technologies. He’ll also spend a bit more time on DocumentDB, Microsoft’s document database-as-a-service, and Azure Search. This session is aimed at anybody who’s new to NoSQL or is curious about the NoSQL options that Azure provides today.
Select the file type you wish to download
Slide Content
  1. NoSQL on Microsoft Azure:An Introduction

    Slide 1 - NoSQL on Microsoft Azure:An Introduction

    • David Chappell
    • Chappell & Associates
    • www.davidchappell.com
    • BRK2555
  2. The Main Idea

    Slide 2 - The Main Idea

    • NoSQL technologies are a fundamental part of Azure today
    • !
  3. Data NowThe world today

    Slide 3 - Data NowThe world today

    • Data is as critical as ever
    • It’s what the people who pay us care most about
    • Data is much more plentiful
    • Storage costs are lower
    • There are bigger data sources:
    • Web-scale applications
    • Internet of Things (IoT)
    • More
    • New data technologies abound
    • NoSQL
    • Big data analytics
    • Search
    • Our field was originally called data processing
    • This isn’t the post-SQL era, but it is the SQL+ era
  4. Data and the CloudIllustrating the intersection

    Slide 4 - Data and the CloudIllustrating the intersection

    • On-Premises
    • Cloud
    • The Trend
    • SQL
    • SQL+
  5. Slide 5

    • Data on Azure:NoSQL Services
  6. Slide 6

    • Operational Data
    • Analytical Data
    • Azure Data TechnologiesA summary
    • Managed service provided by Azure
    • Key/Value Store
    • (Tables, Riak, …)
    • Software that can run in Azure virtual machines
    • NoSQL
    • Technologies
    • SQL
    • Technologies
    • Big Data Analytics
    • (HDInsight, Hadoop)
    • Document Store
    • (DocumentDB, MongoDB, …)
    • Relational Database
    • (SQL Database,
    • SQL Server, Oracle, MySQL, …)
    • Column Family Store
    • (HBase, Cassandra …)
    • Relational Analytics
    • (SQL Server, Oracle, MySQL, …)
  7. Slide 7

    • Tables
    • SQL DatabaseA relational data service
    • Column Name
    • Column Type
    • SQL
    • Query
    • Primary Key
    • Data
    • SQL Database
    • char
    • Name
    • ID
    • int
    • date
    • LastUse
    • char
    • Country
    • Age
    • int
    • 1
    • 3
    • 2
    • 7
    • Application
  8. Technology BasicsSharding

    Slide 8 - Technology BasicsSharding

    • SQL Database Elastic Scale (in preview) now supports sharding
    • Shard 1
    • Shard 2
    • Shard 3
    • Sharded Database
    • Adam
    • Andrew
    • Anusha
    • Bertrand
    • Bill
    • Carl
    • Catherine
    • Cynthia
    • Database
    • Adam
    • Andrew
    • Anusha
    • Bertrand
    • Bill
    • Carl
    • Catherine
    • Cynthia
    • Atomic transactions typically span only a single shard
  9. Slide 9

    • Comparing Azure Managed Data Services
    • SQL Database
    • Category
    • Relational
    • Storage
    • Abstractions
    • Tables,
    • rows, columns
    • Transaction
    • Support
    • All rows and tables in a database
    • Secondary
    • Indexes
    • Yes
    • Pricing
    • Units of throughput
    • Stored Procedure/ Triggers
    • Written in T-SQL
    • Query
    • Language
    • SQL
    • Maximum
    • Database
    • Size
    • 500 GB
    • With Elastic Scale, 100s of TBs
  10. Why Use a NoSQL Technology on Azure?

    Slide 10 - Why Use a NoSQL Technology on Azure?

    • To scale for lots of users and lots of data
    • Pros: NoSQL technologies can offer more scalability than relational databases
    • Cons: Often lose some benefits of relational databases, e.g., database-wide transactions
    • To work better with different data formats, e.g., JSON
    • Pros: Avoiding object/relational mapping makes code easier to write
    • Cons: Limited BI tools;
    • persistent data designed for a single application is harder to share
    • To work with data in a more flexible way
    • Pros: NoSQL technologies don’t have fixed schemas
    • Cons: Fixed schemas help prevent errors
  11. DocumentDBA document store

    Slide 11 - DocumentDBA document store

    • Collections
    • Request
    • Document 1
    • {
    • "name": "John",
    • "country": "Canada",
    • "age": 43,
    • "lastUse": "March 4, 2014"
    • }
    • {
    • "name": "Eva",
    • "country": "Germany",
    • "age": 25
    • }
    • Document 2
    • {
    • "name": "Lou",
    • "country": "Australia",
    • "age": 51,
    • "firstUse": "May 8, 2013"
    • }
    • Document 3
    • {
    • "docCount": 3,
    • "last": "May 1, 2014"
    • }
    • Document 4
    • {…}
    • DocumentDB
    • Application
  12. DocumentDBWays to work with data

    Slide 12 - DocumentDBWays to work with data

    • RESTful access methods
    • For Create/Read/Update/Delete (CRUD) operations
    • DocumentDB SQL
    • A query language with SQL-derived syntax
    • Example:
    • SELECT c.age
    • FROM customers c
    • WHERE c.name = "Lou"
    • Executing logic in the database
    • Stored procedures
    • Triggers
    • User-defined functions (UDFs)
    • Allow extending DocumentDB SQL
    • All written in JavaScript
  13. Slide 13

    • Web Browser
    • JavaScript Application
    • Phone/Tablet
    • Native
    • Apps
    • Microsoft Azure
    • DocumentDBWith Node.js
    • Node.js
    • JavaScript Server Code
    • Web Sites
    • JSON
    • Collection
    • JSON
    • JSON
    • DocumentDB
    • JSON
    • JSON
    • Request
    • JSON
  14. Slide 14

    • Collection
    • Collection
    • Collection
    • Database
    • JSON
    • JSON
    • JSON
    • JSON
    • JSON
    • JSON
    • JSON
    • JSON
    • Collection
    • JSON
    • JSON
    • JSON
    • DocumentDBSharding and transactions
    • Atomic transactions can span only a single collection
    • The unit of sharding is a collection
  15. Slide 15

    • Database
    • Technology BasicsReplication and consistency
    • Shard A
    • What does a reader see?
    • A write to the primary replica takes time to propagate to the secondaries
    • Primary replica
    • Secondary replica
    • Shard A
    • Shard A
    • Replication can improve performance and availability
  16. DocumentDBConsistency options

    Slide 16 - DocumentDBConsistency options

    • Session
    • Strong
    • Bounded Staleness
    • Yes, but only for writes by other clients
    • Yes, but only within a specified interval
    • No
    • Eventual
    • Moderately slow
    • No
    • Readers might see out-of-order updates
    • No
    • Readers might see old data
    • Slowest
    • Speed of reads
    • Speed of writes
    • Slowest
    • Fastest
    • Yes
    • Yes, but only for writes by other clients
    • Yes
    • Fastest
    • Fastest
    • Moderately fast
    • Fastest
    • The default
  17. Slide 17

    • Pricing
    • Transaction
    • Support
    • Secondary
    • Indexes
    • Comparing Azure Managed Data Services
    • SQL Database
    • Storage
    • Abstractions
    • Category
    • DocumentDB
    • Relational
    • Document store
    • Tables,
    • rows, columns
    • Collections, documents
    • All rows and tables in a database
    • Yes
    • Yes
    • Units of throughput
    • Units of throughput
    • All documents in the same collection
    • Stored Procedures/ Triggers
    • Written in T-SQL
    • Written in JavaScript
    • Query
    • Language
    • SQL
    • Extended subset of SQL
    • Maximum
    • Database
    • Size
    • 500 GB
    • 100s of TBs
  18. Slide 18

    • Tables
    • Partition
    • B
    • Partition
    • A
    • Property
    • TablesA key/value store
    • Entity
    • 2
    • B
    • Property Type
    • Data
    • Property Name
    • String
    • int
    • String
    • Date
    • Name
    • LastUse
    • Country
    • Age
    • String
    • int
    • String
    • Name
    • Country
    • Age
    • String
    • int
    • String
    • Date
    • Name
    • FirstUse
    • Country
    • Age
    • Row key
    • 1
    • 2
    • 1
    • 2
    • Partition key
    • A
    • A
    • B
    • B
    • int
    • Date
    • Last
    • Count
    • 2
    • B
    • Azure Tables
    • Application
  19. TablesSharding and transactions

    Slide 19 - TablesSharding and transactions

    • Partitions are replicated; reads and writes provide strong consistency
    • Table
    • Partition A
    • Partition B
    • Partition C
    • A
    • 1
    • A
    • 2
    • A
    • 3
    • B
    • 1
    • B
    • 2
    • B
    • 3
    • C
    • 1
    • C
    • 2
    • C
    • 3
    • Atomic transactions can span only a single partition
    • The unit of sharding is a partition
  20. Slide 20

    • Pricing
    • Transaction
    • Support
    • Secondary
    • Indexes
    • Comparing Azure Managed Data Services
    • SQL Database
    • Tables
    • Storage
    • Abstractions
    • Category
    • DocumentDB
    • Relational
    • Key/value store
    • Document store
    • Tables,
    • rows, columns
    • Collections, documents
    • All rows and tables in a database
    • Yes
    • No
    • Yes
    • Units of throughput
    • GBs of storage
    • Units of throughput
    • Tables,
    • partitions,
    • entities
    • All entities in the same partition
    • All documents in the same collection
    • Stored Procedures/ Triggers
    • Written in T-SQL
    • None
    • Written in JavaScript
    • Query
    • Language
    • SQL
    • Subset of OData queries
    • Extended subset of SQL
    • Maximum
    • Database
    • Size
    • 500 GB
    • 100s of TBs
    • 100s of TBs
  21. Slide 21

    • Tables
    • HDInsight HBaseA column family store
    • Column Key (Qualifier)
    • Column Key (Family)
    • 2
    • LastUse
    • v2
    • Usage
    • User
    • Usage
    • Name
    • Country
    • LastUse
    • FirstUse
    • Row
    • Key
    • Age
    • Data (optionally with time-stamped versions)
    • 1
    • 2
    • 3
    • 5
    • 6
    • 4
    • HDInsight HBase
    • Application
    • HDInsight supports Phoenix for SQL queries on HBase
  22. HDInsight HBaseSharding and transactions

    Slide 22 - HDInsight HBaseSharding and transactions

    • Table
    • Region A
    • Region B
    • Region C
    • The unit of sharding is a region
    • Regions are replicated; reads and writes provide strong consistency
    • HBase automatically shards a table; users don’t see regions
    • Atomic transactions can span only a single row
  23. Slide 23

    • Pricing
    • Transaction
    • Support
    • Secondary
    • Indexes
    • Comparing Azure Managed Data Services
    • SQL Database
    • Tables
    • Storage
    • Abstractions
    • Category
    • DocumentDB
    • Relational
    • Key/value store
    • Document store
    • Tables,
    • rows, columns
    • Collections, documents
    • All rows and tables in a database
    • Yes
    • No
    • Yes
    • Units of throughput
    • GBs of storage
    • Units of throughput
    • Tables,
    • partitions,
    • entities
    • All entities in the same partition
    • All documents in the same collection
    • Stored Procedures/Triggers
    • Written in T-SQL
    • None
    • Written in JavaScript
    • HDInsight HBase
    • Column family store
    • Tables, rows, columns, cells, column families
    • No
    • GBs of storage plus VMs per hour
    • All cells in the same row
    • Written in Java
    • Query
    • Language
    • SQL
    • Subset of OData queries
    • Extended subset of SQL
    • SQL subset w/ Phoenix
    • Maximum
    • Database
    • Size
    • 500 GB
    • 100s of TBs
    • 100s of TBs
    • 100s of TBs
  24. Slide 24

    • Hadoop Technologies
    • HDFS
    • Big Data Analytics (and More)The Hadoop technology family
    • HBase
    • YARN
    • Azure HDInsight provides these as a managed service
    • Storm
    • MapReduce
    • . . .
    • Tez
    • Hive, Pig, …
  25. Slide 25

    • Blob
    • 100011010011001111011111011011010001101
    • Blob
    • 100011010011001111011111011011010001101
    • Blob
    • 100011010011001111011111011011010001101
    • Azure Blobs
    • HDInsight Tez/MapReduceAn illustration
    • HDFS API
    • Excel
    • Tez/ MapReduce Job
    • Logic
    • Logic
    • Logic
    • Hive
    • Pig
    • . . .
    • VM
    • VM
    • VM
    • HDInsight
    • HDInsight HBase is also implemented on this API and relies on Azure Blobs
    • . . .
    • . . .
  26. Slide 26

    • Data on Azure:Search Services
  27. Operational Data and Search DataDifferent goals, different technologies

    Slide 27 - Operational Data and Search DataDifferent goals, different technologies

    • Operational
    • Data
    • Relational/ NoSQL Store
    • Index
    • Azure
    • Search
    • Index
    • Search requests
    • Search results
    • Read/write requests
    • Results
    • Application
    • User
  28. Slide 28

    • Online retailer
    • Example: An online shoe store
    • User-generated content site
    • Example: A discussion site for movie buffs
    • Custom business application
    • Example: An employee benefits application
    • Search ScenariosSome examples
  29. Slide 29

    • Example Search BoxFor an online shoe retailer
    • high
    • high heels
    • high tops
    • high arch
    • Users expect suggestions
    • Azure Search doesn’t provide any UI components
  30. Slide 30

    • Example Search Results: TextFor an online shoe retailer
    • Contoso Brand High Heels
    • Pumps, stilettos, and more
    • Fabrikam’s Fancies
    • High heels for fun!
    • High Heels for Everybody
    • Fashion, fashion, and more fashion
    • Search terms shown in bold
    • Results returned in a specific order
  31. Slide 31

    • Example Search Results: Text and GraphicsFor an online shoe retailer
    • Price
    • $100 or less (10)
    • $100 - $250 (14)
    • $250 and up (9)
    • Color
    • Contoso
    • Pumps
    • $129.95
    • Fabrikam
    • Flower
    • $59.99
    • Fabrikam
    • Lipstick Heels
    • $489.95
    • Contoso
    • Stilettos
    • $350.00
    • (34)
    • (22)
    • (19)
    • (11)
    • (21)
    • (5)
    • (9)
    • (10)
    • Help with the user’s next search
    • Requires more information than a simple text UI
  32. Using Azure SearchCreating an index

    Slide 32 - Using Azure SearchCreating an index

    • Azure
    • Search
    • Create index
    • Schema
    • Application
    • Index
  33. Slide 33

    • An Example SchemaFor an online shoe retailer
    • Category String Searchable, Suggestions, Sortable, Retrievable, Filterable
    • Style String Searchable, Suggestions, Sortable, Retrievable, Filterable
    • Color Collection(String) Searchable, Suggestions, Retrievable, Filterable, Facetable
    • Brand String Searchable, Suggestions, Sortable, Retrievable, Filterable
    • Price Double Searchable, Sortable, Retrievable, Filterable, Facetable
    • Picture String Retrievable
    • Stock Int32
    • Promotion Boolean
    • Name Type Other Attributes
    • For ordering search results
    • Holds a URL to an Azure blob
  34. Using Azure SearchPopulating an index

    Slide 34 - Using Azure SearchPopulating an index

    • Azure
    • Search
    • Provide data
    • Application
    • Index
  35. Slide 35

    • False
    • False
    • False
    • True
    • Contoso
    • Boots
    • High Heels
    • High Heels
    • Tuxedo
    • Fabrikam
    • Fabrikam
    • Fabrikam
    • http://...
    • http://...
    • http://...
    • http://...
    • 100
    • 10
    • 158
    • 34
    • $134.79
    • $489.95
    • $59.99
    • $500.00
    • White, Black
    • Pink
    • Red
    • Black
    • Beatle
    • Lipstick
    • Heels
    • Flower
    • Black
    • . . .
    • . . .
    • . . .
    • . . .
    • . . .
    • . . .
    • . . .
    • . . .
    • An Example IndexFor an online shoe retailer
    • Category
    • Brand
    • Style
    • Price
    • Picture
    • Stock
    • Promotion
    • Color
    • Sneakers
    • Contoso
    • White,
    • Black
    • $29.95
    • http://...
    • 194
    • True
    • HiTops
    • Contoso
    • True
    • High Heels
    • http://...
    • 285
    • $129.95
    • Red
    • Pumps
    • Contoso
    • True
    • High Heels
    • http://...
    • 23
    • $350.00
    • Red,
    • Black
    • Stilettos
  36. Slide 36

    • Price
    • $100 or less
    • $100 - $250
    • $250 and up
    • Color
    • Contoso
    • Pumps
    • $129.95
    • Fabrikam
    • Flower
    • $59.99
    • Fabrikam
    • Lipstick Heels
    • $489.95
    • Contoso
    • Stilettos
    • $350.00
    • Using Azure SearchSearching
    • Azure Search
    • Index
    • User
    • text=“high heels”
    • Search Request
    • Application
    • Search Result
    • high heels
  37. Slide 37

    • False
    • False
    • False
    • True
    • Contoso
    • Boots
    • High Heels
    • High Heels
    • Tuxedo
    • Fabrikam
    • Fabrikam
    • Fabrikam
    • http://...
    • http://...
    • http://...
    • http://...
    • 100
    • 10
    • 158
    • 34
    • $134.79
    • $489.95
    • $59.99
    • $500.00
    • White, Black
    • Pink
    • Red
    • Black
    • Beatle
    • Lipstick
    • Heels
    • Flower
    • Black
    • . . .
    • . . .
    • . . .
    • . . .
    • . . .
    • . . .
    • . . .
    • . . .
    • Search Results: Example For an online shoe retailer
    • Sneakers
    • Contoso
    • White,
    • Black
    • $29.95
    • http://...
    • 194
    • True
    • HiTops
    • Contoso
    • True
    • High Heels
    • http://...
    • 285
    • $129.95
    • Red
    • Pumps
    • Contoso
    • True
    • High Heels
    • http://...
    • 23
    • $350.00
    • Red,
    • Black
    • Stilettos
    • Returned by a search for “high heels”
    • Category
    • Brand
    • Style
    • Price
    • Picture
    • Stock
    • Promotion
    • Color
  38. Slide 38

    • Price
    • $100 or less (10)
    • $100 - $250 (14)
    • $250 and up (9)
    • Color
    • Contoso
    • Pumps
    • $129.95
    • Fabrikam
    • Flower
    • $59.99
    • Fabrikam
    • Lipstick Heels
    • $489.95
    • Contoso
    • Stilettos
    • $350.00
    • (34)
    • (22)
    • (19)
    • (11)
    • (21)
    • (5)
    • (9)
    • (10)
    • Search Results: UIFor an online shoe retailer
    • Made possible by facets
    • Contoso is first because of the promotion
    • Pumps are first because there are more in stock
  39. Slide 39

    • Data on Azure:The Big Picture
  40. Slide 40

    • Working with DataThe traditional world
    • Analytical Data
    • Relational
    • Data Warehouse
    • BI Applications
    • Internal Users
    • Operational Data
    • Relational
    • Database
    • End Users
    • Applications
  41. Slide 41

    • Search Data
    • Indexes
    • Analytical Data
    • Unstructured
    • Data
    • Relational
    • Data Warehouse
    • Working with DataA modern view
    • End Users
    • Operational Data
    • Relational
    • Database
    • NoSQL
    • Store
    • Internal Users
    • BI Applications
    • Applications
  42. Slide 42

    • Working with DataMicrosoft Azure technologies
    • Operational Data
    • Relational
    • Database
    • NoSQL
    • Store
    • Analytical Data
    • Search Data
    • Indexes
    • Unstructured
    • Data
    • Relational
    • Data Warehouse
    • HDInsight Hadoop
    • DocumentDB, Tables, HDInsight HBase
    • Search
    • Managed service provided by Azure
    • Software that can run in Azure virtual machines
    • SQL Server in an IaaS VM
    • SQL Server in an IaaS VM, SQL Database
  43. Choosing a Data TechnologyOptions in the new world

    Slide 43 - Choosing a Data TechnologyOptions in the new world

    • Use SQL Database when:
    • You want relational data
    • You want to get your application up as fast as possible
    • You want your application to require minimal management
    • Use DocumentDB, Tables, or HDInsight HBase when:
    • You need more scale than relational allows
    • You want a non-relational data model
    • You don’t want to be locked into a schema
    • Use Azure Search when:
    • You want to provide a search interface to your users
    • Use HDInsight Hadoop Tez/MapReduce when:
    • You want to do big-data analytics
  44. Slide 44

    • Operational data: DocumentDB, Tables, HDInsight HBase
    • Analytical data: HDInsight Hadoop
    • Search: Azure Search
    • Welcome to the SQL+ era
    • Conclusions
    • NoSQL technologies are a fundamental part of Azure today
    • !
  45. Slide 45

    • About the Speaker
    • David Chappell is Principal of Chappell & Associates (www.davidchappell.com) in San Francisco, California. Through his speaking, writing, and consulting, he helps people around the world understand, use, and make better decisions about new technology. David has been the keynote speaker for more than a hundred events and conferences on five continents, and his seminars have been attended by tens of thousands of IT leaders, architects, and developers in fifty countries. His books have been published in a dozen languages and used regularly in courses at MIT, ETH Zurich, and other universities. In his consulting practice, he has helped clients such as Hewlett-Packard, IBM, Microsoft, Stanford University, and Target Corporation adopt new technologies, market new products, and educate their customers and staff.
  46. Slide 46

    • Ignite Azure Challenge Sweepstakes
    • Attend Azure sessions and activities, track your progress online, win raffle tickets for great prizes!
    • Aka.ms/MyAzureChallenge
    • Enter this session code online: BRK2555
    • NO PURCHASE NECESSARY. Open only to event attendees. Winners must be present to win. Game ends May 9th, 2015. For Official Rules, see The Cloud and Enterprise Lounge or myignite.com/challenge