2-735: Building Rich, Contextually Aware Universal Windows Apps Using Sensors

Want to build rich, contextually-aware apps with sensors for Windows 10? Learn how the Windows Sensors APIs can help you build smart apps for your daily life: apps that detect your presence when you approach your device; apps that figure out your activities – whether you are walking, running, driving, biking, and helps you track your fitness routines. See demos on app scenarios including health and fitness, navigation, and power saving using with Activity and Environmental sensors. Learn about the Windows 10 sensor features, custom sensors for hardware innovation, and reusing your Android and iOS app code to build Universal Windows apps.

1.0x

2-735: Building Rich, Contextually Aware Universal Windows Apps Using Sensors

Created 2 years ago

Duration 1:00:45
lesson view count 843
Want to build rich, contextually-aware apps with sensors for Windows 10? Learn how the Windows Sensors APIs can help you build smart apps for your daily life: apps that detect your presence when you approach your device; apps that figure out your activities – whether you are walking, running, driving, biking, and helps you track your fitness routines. See demos on app scenarios including health and fitness, navigation, and power saving using with Activity and Environmental sensors. Learn about the Windows 10 sensor features, custom sensors for hardware innovation, and reusing your Android and iOS app code to build Universal Windows apps.
Select the file type you wish to download
Slide Content
  1. Lisa Ong                                                        Rinku Sreedhar

    Slide 1 - Lisa Ong Rinku Sreedhar

    • Principal Software Engineer Senior Program Manager
    • Sensors Sensors
    • Building Rich, Contextually Aware Applications Using Sensors
    • 2-735
  2. Promise of Context Awareness

    Slide 2 - Promise of Context Awareness

    • Windows 10 Sensor APIs
    • Demos
    • Agenda slide
  3. Promise of Context Awareness

    Slide 3 - Promise of Context Awareness

    • Set of facts about users
    • Reduce explicit interaction, more responsive
    • Advanced inferences through machine learning
    • Running, working out
    • Running, late for a meeting
    • High altitude biking
    • Walking, Shopping
    • In car, Driving, Stopped
    • Sleeping
  4. How Sensors enable Context Awareness

    Slide 4 - How Sensors enable Context Awareness

    • Are you near your device?
    • Are you driving? Or walking or biking?
    • Are you trying to track to a fitness goal?
    • Did you forget where you parked?
  5. Introducing the Contextual Awareness Sensor API

    Slide 5 - Introducing the Contextual Awareness Sensor API

  6. Slide 6

    • WinRT API to detect proximity state, distance and range
    • Supports Short and Long Range Sensors
    • Applications
    • Automatic display off during a phone call
    • Wake on approach
    • In pocket detection
    • Gestures
    • Proximity API
  7. Slide 7

    • WinRT API to detect proximity state, distance and range
    • Supports Short and Long Range Sensors
    • Applications
    • Automatic display off during a phone call
    • Wake on approach
    • In pocket detection
    • Gestures
    • Proximity API
  8. Proximity DemoSurface Hub

    Slide 8 - Proximity DemoSurface Hub

  9. Sensors API namespace

    Slide 9 - Sensors API namespace

    • using Windows.Devices.Sensors;
    • Getting the default sensor (most sensor APIs only return one sensor on the system )
    • Barometer barometer = Barometer.GetDefault();
    • Getting the sensor (when there are multiple sensors – e.g.: proximity and Custom Sensors)
    • DeviceWatcher watcher =
    • DeviceInformation.CreateWatcher(ProximitySensor.GetDeviceSelector());
    • watcher.Added += OnProximitySensorAdded;
    • . . .
    • void OnProximitySensorAdded(DeviceWatcher sender, DeviceInformation device)
    • {
    • ProximitySensor sensor= ProximitySensor.FromId(device.Id);
    • }
    • Intro to Sensors API Pattern using Proximity
  10. Slide 10

    • API to detect user’s motion context
    • Run, Walk, On bicycle, In vehicle, Fidgeting, Stationary, Idle, Unknown
    • Most likely activity with confidence
    • 30 day history
    • Typical Applications
    • Health & fitness tracking
    • Navigation, Maps, Safe driving
    • Power Saving
    • Activity Detection API
  11. DemoMS Research Predestination Appusing Simulator

    Slide 11 - DemoMS Research Predestination Appusing Simulator

  12. Using Activity Sensors and Background Triggers

    Slide 12 - Using Activity Sensors and Background Triggers

    • Getting the current activity
    • var reading = await activitySensor.GetCurrentReadingAsync();
    • Subscribing to activity changes
    • activitySensor.ReadingChanged += new TypedEventHandler<ActivitySensor,
    • ActivitySensorReadingChangedEventArgs>(ReadingChanged);
    • Getting history of up to 30 days
    • DateTimeOffset yesterday = ...
    • var history = await ActivitySensor.GetSystemHistoryAsync(yesterday);
    • foreach (var entry in history) { ... }
    • Using a background task
    • var trigger = new
    • Windows.ApplicationModel.Background.ActivitySensorTrigger(reportIntervalMs);
    • trigger.SubscribedActivities.Add(ActivityType.InVehicle);
    • // .. register the trigger, etc..
  13. Using Activity Sensors and Background Triggers

    Slide 13 - Using Activity Sensors and Background Triggers

    • Getting the current activity
    • var reading = await activitySensor.GetCurrentReadingAsync();
    • Subscribing to activity changes
    • activitySensor.ReadingChanged += new TypedEventHandler<ActivitySensor,
    • ActivitySensorReadingChangedEventArgs>(ReadingChanged);
    • Getting history of up to 30 days
    • DateTimeOffset yesterday = ...
    • var history = await ActivitySensor.GetSystemHistoryAsync(yesterday);
    • foreach (var entry in history) { ... }
    • Using a background task
    • var trigger = new
    • Windows.ApplicationModel.Background.ActivitySensorTrigger(reportIntervalMs);
    • trigger.SubscribedActivities.Add(ActivityType.InVehicle);
    • // .. register the trigger, etc..
  14. Using Activity Sensors and Background Triggers

    Slide 14 - Using Activity Sensors and Background Triggers

    • Getting the current activity
    • var reading = await activitySensor.GetCurrentReadingAsync();
    • Subscribing to activity changes
    • activitySensor.ReadingChanged += new TypedEventHandler<ActivitySensor,
    • ActivitySensorReadingChangedEventArgs>(ReadingChanged);
    • Getting history of up to 30 days
    • DateTimeOffset yesterday = ...
    • var history = await ActivitySensor.GetSystemHistoryAsync(yesterday);
    • foreach (var entry in history) { ... }
    • Using a background task
    • var trigger = new
    • Windows.ApplicationModel.Background.ActivitySensorTrigger(reportIntervalMs);
    • trigger.SubscribedActivities.Add(ActivityType.InVehicle);
    • // .. register the trigger, etc..
  15. Slide 15

    • API to detect users steps
    • Number of steps – walking and running
    • Active time spent walking and running
    • 30 day history
    • Typical Applications
    • Health & Fitness Tracking – no fitness gadget required
    • Combine wearables and phone sensor data
    • Pedometer API
  16. Using Pedometer API

    Slide 16 - Using Pedometer API

    • Getting the pedometer reading changes
    • pedometer.ReadingChanged += new TypedEventHandler<Pedometer,
    • PedometerReadingChangedEventArgs>(ReadingChanged);
    • void ReadingChanged(
    • Pedometer sender, PedometerReadingChangedEventArgs args)
    • {
    • PedometerReading reading = args.Reading;
    • if (reading.StepKind == PedometerStepKind.Walking)
    • walkingSteps = reading.CumulativeSteps;
    • }
    • Getting pedometer step count history
    • var history = await Pedometer.GetSystemHistoryAsync(yesterday);
  17. Demo Pedometer on Desktop and Mobile

    Slide 17 - Demo Pedometer on Desktop and Mobile

  18. Barometer API reports barometric station pressure

    Slide 18 - Barometer API reports barometric station pressure

    • Altimeter API reports relative altitude/ elevation
    • Typical Applications
    • Health & fitness
    • Floor Sensing and Indoor Navigation
    • Weather Forecasting
    • Barometer / Altimeter API
  19. Using Barometer/Altimeter API

    Slide 19 - Using Barometer/Altimeter API

    • Same Sensor API patterns….
    • Barometer barometer = Barometer.GetDefault();
    • BarometerReading reading = barometer.GetCurrentReading();
    • double pressure = reading.StationPressureInHectopascals;
    • barometer.ReadingChanged += ...
    • Altimeter altimeter = Altimeter.GetDefault();
    • AltimeterReading altimeterReading = altimeter.GetCurrentReading();
    • double altitudeChange = altimeterReading.AltitudeChangeInMeters;
    • altimeter.ReadingChanged += ...
    • Select a Report Interval
    • mySensor.ReportInterval = 500;
  20. Slide 20

    • Add a completely new Sensor Examples: CO2 Sensor, UV Sensor, Heart Rate Sensor…
    • Hardware vendors can introduce new Sensor Types independent of Microsoft OS releases
    • Simple, familiar API for reading sensor data
    • Custom Sensor API
  21. Slide 21

    • Add a completely new Sensor Examples: CO2 Sensor, UV Sensor, Heart Rate Sensor…
    • Hardware vendors can introduce new Sensor Types independent of Microsoft OS releases
    • Simple, familiar API for reading sensor data
    • Custom Sensor API
  22. Extensibility using Custom Sensors

    Slide 22 - Extensibility using Custom Sensors

    • Step 1: Driver defines and implements sensor IDs and keys
    • // Driver-defined IDs: UV sensor and UV level reading
    • Guid UVSensorID = new Guid("4025a865-638c-43aa-a688-98580961eeae");const String UVLevelKey = "{74879888-a3cc-45c6-9ea9-058838256433} 1";
    • Step 2: App finds that custom sensor
    • const selector = CustomSensor.GetDeviceSelector(UVSensorID);
    • watcher = Windows.Devices.Enumeration.DeviceInformation.CreateWatcher(selector);
    • … and consumes its data
    • CustomSensorReading reading = sensor.GetCurrentReading();
    • if (reading.Properties.ContainsKey(UVLevelKey))
    • float UVLevel = reading.Properties[UVLevelKey];
    • sensor.ReadingChanged += …
  23. Buffers sensor samples to a batch

    Slide 23 - Buffers sensor samples to a batch

    • Client specifies the report latency
    • accelerometer.ReportLatency =
    • accelerometer.MaxBatchSize * desiredReportInterval;
    • Typical Applications:
    • Power saving features
    • Background scenarios like Sleep and Activity monitoring
    • Sensor Batching
  24. Slide 24

    • Developer Proposition
    • Reuse Existing Android/ iOS Code to Build UWP Apps
    • Reusing Android Code – Sensors Supported
    • Activity Detection
    • Screen Rotation
    • Accelerometer
    • Gyroscope
    • Magnetometer
    • Proximity
    •  Reusing iOS Code – Sensors Supported
    • Accelerometer
    • Gyroscope
    • Magnetometer
    • Reusing Android and iOS app/code
    • Windows Bridges
    • ‘Project Astoria’ (Java/C++)
    • ‘Project Islandwood’ (Objective C/C++)
  25. Slide 25

    • Lumia SensorCore APIs – released in 2014 with Activity Tracking features on Lumia Phones
    • Windows 10 now supports these features on devices ranging from mobile, desktop, IOT
    • All future enhancement for Activity Detection and Pedometer features will be on the UWP APIs
    • Guidelines to developers :
    • For new app development – use UWP APIs
    • To target Legacy Lumia phones - use SensorCore APIs
    • (Note: Places and Track features of SensorCore is Out of Scope)
    • Lumia SensorCore API
  26. Partner Demo MS Health

    Slide 26 - Partner Demo MS Health

  27. Partner Demo MS Health

    Slide 27 - Partner Demo MS Health

  28. Additional Tips to know

    Slide 28 - Additional Tips to know

    • ReadingTransform
    • Rotates sensor data for you to a display orientation
    • 1 line change to port apps from portrait-first devices to landscape-first devices (vice-versa)
    • Data is streamed consistently across platforms
    • UWP apps on multiple platforms get the same behavior for ReportInterval
    • Using Simulator to build and Test your app
    • Download universal driver samples from Github to emulate new sensor types
    • Privacy
    • Users can control privacy via Privacy Settings
  29. Additional Tips to know

    Slide 29 - Additional Tips to know

    • ReadingTransform
    • Rotates sensor data for you to a display orientation
    • 1 line change to port apps from portrait-first devices to landscape-first devices (vice-versa)
    • Data is streamed consistently across platforms
    • UWP apps on multiple platforms get the same behavior for ReportInterval
    • Using Simulator to build and Test your app
    • Download universal driver samples from Github to emulate new sensor types
    • Privacy
    • Users can control privacy via Privacy Settings