TechEd 2008 Developers – A Mi Gente de Southworks (ES)

By Zach Skyles Owens (Perdona mi español)

Good times backstage - me, Dave, Zach, Johnny, Lito, and TimAcabo de escribir como la dedicación del equipo de desarrollo fue el clave a nuestro éxito. Aparte de los ingenieros a Microsoft trabajamos con Southworks que es un grupo de gente muy apasionante de su trabajo y de tecnología.

Este proyecto requerido mucha dedicación porque no teníamos mucho tiempo y teníamos que desarrollar mucha tecnología. La mayoría del equipo estaba en Buenos Aires pero esto no fue un problema porque manteníamos comunicación muy abierta y tenían líderes muy efectivos. La verdad es que el equipo trabó en una fiesta porque teníamos que presentar el demo al equipo de BillG el día siguiente. Muchas gracias a las familias de ellos por tener la paciencia.

A mi gente de Southworks… Les saludo y tienen el respecto de muchos en Redmond!

TechEd 2008 Developers – Data Platform Keynote EDM/SSDS/SQL Data Types

By Zach Skyles Owens

One of the biggest challenges we faced when designing this demo was trying to make the breadth of Microsoft’s Data Platform technologies easily understandable in 5 minutes.  The major architecture components are SQL Server 2008, SQL Server Data Services (SSDS), SQL Server Compact and Sync; which I outlined in a previous post.  We’ve been following the blogosphere as closely as possible and, understandably so, one of the areas we’ve seen a bit of confusion around is the type support and conversion between the Entity Data Model, SSDS and SQL Server 2008.

There are a lot of moving parts here so I’ll do my best at explaining how everything was integrated and how we got around some of the differences in type support of the data platform technologies.

Web Application

image Bloggers use this application to submit geo-tagged articles and images which are stored in SSDS.  We embedded a Virtual Earth control which the users use to manually geo-tag their content.  Those who have been following SSDS closely may have noted that SSDS does not currently support blob storage or spatial types. 

  • Blob Storage in SSDS – This is the number one feature that customers are asking for right now with SSDS.  Until SSDS supports blobs our current workaround was to Base64 Encode the images for storage in SSDS.
  • Spatial Storage in SSDS – Currently SSDS does not have support for Spatial types.  This was an easy work around for us by converting the spatial POINT coordinates to the Well Known Text (WKT) format and storing them as text. 

Mobile Application

image In this scenario we are also using a Windows Mobile device that allows bloggers to submit photos which are automatically sent to SSDS via the Microsoft Sync Framework.  Here are some key points:

  • A SQL Server Compact database stores the application’s meta-data
  • Images are stored on the device’s file system
  • Geo-tagging is pulled directly from the GPS enabled device.  The app caches the last known GPS coordinates in case of lack of GPS connectivity.
  • The Sync provider running on the device converts the geo-data to WKT and the image to Base64 text as in the web app

Sync from SSDS to SQL Server

sync2 Our application uses a powerful WPF desktop application connected to a local SQL Server 2008 database.  Since SQL Server 2008 supports Spatial we have the ability to run high performance spatial queries which aren’t currently possible in SSDS.  FILESTREAM allows us to use the high-performance of the file system for binary file storage along with transactional consistency and great manageability of the database.  Type conversion here was very simple and outlined below:

  • Sync and FILESTREAM – The Sync Provider sitting on our SQL Server database pulls the Base64 binary data from SSDS and inserts it into a varbinary(max) FILESTREAM column in our database.
  • Sync and Spatial – This Sync Provider inserts the WKT POINT data into a SQL Server GEOGRAPHY type.

EDM, Spatial and FILESTREAM

  imageimageOur WPF application uses the Entity Data Model (EDM) to provide the application developers with a more natural business representation of the data. This allows the database model and application data model to evolve independently. Currently the EDM has limited support for FILESTREAM and does not natively support the new SQL Server 2008 Spatial types (GEOGRAPHY and GEOMETRY).  These were also very easy to work around in the following ways.

  • EDM and FILESTREAM – Currently the EDM treats FILESTREAM in the same way it treats any other varbinary(max) column.  You get the transaction consistency and manageability of the database.  It interacts with the FILESTREAM data through T-SQL so you don’t get the Win32 streaming performance that FILESTREAM has the ability to provide.  If that type of read/write performance is needed you can easily write a section of code that interacts with traditional database connections and SQL.
  • EDM and Spatial – Currently the EDM does not support the new Spatial types.   Our WPF application had two main requirements for Spatial: high performance queries and showing the Spatial meta-data.  We wrote a stored procedure for the queries and mapped a calculated column which converted the spatial data into WKT for displaying the meta-data.

Summary

There were definitely some things that we had to consider when building this application but in the end none were major barriers.  The application works great and was an enjoyable development experience. 

TechEd 2008 Developers – Data Platform Keynote Demo Architecture

By Zach Skyles Owens

Just minutes ago at TechEd 2008 Developers in Orlando Dave Campbell, Technical Fellow, was on stage with Bill Gates doing the Data Platform demo which included a host of exciting Microsoft Technologies.

  • SQL Server 2008 – Supports any data
  • SQL Server Data Services (SSDS) – Quickly provisioning for unpredictable scale
  • Microsoft Sync Framework – Keeping all data synchronized

Building this demo has been one of the most rewarding experiences in my career.  Everyone involved has put in 110%.

When we set out to build this demo we spent a lot of time making sure that the architecture was something that we have heard customers asking for.  Since SSDS is such a new piece of the Microsoft Data Platform going through this process was an interesting experience.

In this article I will quickly highlight the architecture of this application and describe why we made some of the architecture decisions.

Scenario

This application was built around a fictitious company called Trey Research.  Trey Research is a news agency that has launched a new strategy aimed at turning bloggers into paid journalists by paying them for their articles and photos.  Bloggers from around the world submit articles and images through either a web application or Windows Mobile app.  New analysts at Trey Research find the best articles and images for a given geographic area of interest, combine them into a story and sell them to companies like MSNBC; paying the content creators in the process.

Architecture

Here is a high-level overview of the architecture.

image

 

SQL Server 2008 and WPF

imageSo starting from the News Analyst WPF App and SQL Server 2008.  We chose to use local SQL Server database connected to a WPF desktop application for a number of reasons including.

  • High Performance of a Local Database
  • Powerful Analytics of the SQL Server platform
  • Ability to execute Spatial Queries to search for relevant content
  • Storage of all content types including geo-tagged text and images

News Analysts at Trey Research can search for content with a power UI including Spatial and Time based queries.  SQL Server uses powerful analytics on the back end to determine the target demographic for the content.  Some of the exciting technologies being used are:

  • FILESTREAM to store binary data
  • New DateTimeOffset type used to store the date and time a photo was taken, which preserves time zone information
  • Spatial queries and indexes which allow for the fast retrieval of geo-tagged data.
  • SQL Server Reporting Services to provide rich visualization of analytical data.

SQL Server Data Services, Web Application and Mobile Device

imageAll of this content is submitted via a web site or via a Windows Mobile application.  The content submitted via the web site or mobile app are stored in SSDS.  Their core competency is managing news content, not building Internet scale data centers.  Trey Research decided to use SSDS as the data storage platform for a number of reasons:

  • To quickly provision for the unpredictable scale of what is turning out to be a very popular web site
  • The nature of news is that a large event can produce huge spikes in traffic, so they are relying on Microsoft’s global infrastructure and Service Level Agreements
  • SSDS is acting as a Data Hub where web and mobile devices submit content

Microsoft Sync Framework

The final piece of magic in this application is the Microsoft Sync Framework.  Sync is used to pull data down from SSDS into the local SQL Server database in addition to moving data from the mobile device into SSDS.  The sync framework provides a powerful platform for dealing with things like conflict detection, etc.

Summary

As you can see there were a number of architecture decisions that had to be made.  Trey Research is using many of the powerful features of the Microsoft Data Platform to quickly build this application.  I’ll continue to post information about this demo.