Happy New Year 2016 around the World Behind the scenes of my #FsAdvent project

Just like last year and the year before, I wanted to participate in the #FsAdvent event, where someone writes a blog post about something they did with F# during December. Thanks to Sergey Tihon for the organization of the English version and the Japanese F# community for coming up with the idea a few years ago!

As my blog post ended up on 31 December, I wanted to do something that would fit well with the theme of ending of 2015 and starting of the new year 2016 and so I decided to write a little interactive web site that tracks the "Happy New Year" tweets live across the globe. This is partly inspired by Happy New Year Tweets from Twitter in 2014, but rather than analyzing data in retrospect, you can watch 2016 come live!

Without further ado, here are the important links:

Happy New Year 2016 around the World live web site!

(It will stay alive for a few days around 31 December 2015, but not forever.)

(It will stay alive for a few days around 31 December 2015, but not forever.) F# source code for the project on GitHub

(Feel free to modify it and use it for other events!)

(Feel free to modify it and use it for other events!) Continute reading if you want to learn about how it works!

Before we get to the technical details, here is a brief screenshot showing the project live:

Overview

On the front-end side, the web site displays three different things - it shows live tweets on a map, it shows live tweets in a feed (below on the right) and it shows a word cloud with most common phrases. Everything is updated live using a three web socket connections with the server.

On the back-end side, the server uses Twitter Streaming API to receive "Happy New Year" tweets as they happen. It then uses various techniques for getting locations of some tweets so that they can appear on the map and it calculates statistics (e.g. for the word cloud) on the fly.

If you look at the source code, pretty much all back-end is implemented in a single F# script file. For the front-end, I didn't do anything fancy and hacked together some JavaScript using the great D3-based Datamaps library for the map.

There are a couple of nice things in the code including the connection to Twitter, F# type providers (as always), agents for reactive programming and Suave web server for implementing web sockets.

Getting a stream of tweets

To get the tweets, I'm using the F# Data Toolbox library, which comes with a nice Twitter API wrapper built using F# type providers. As a single-user application (all is happening on the server), we can directly provider the application access token & secret and connect to the Twitter directly. Then we can use the twitter.Streaming.FilterTweets method to search for tweets that contain any of the known "Happy New Year" phrases:

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: let phrases = [ "새해 복 많이 받으세요" ; "สวัสดีปีใหม่" ; "šťastný nový rok" ; "عام سعيد" ; (...) ] let ctx = (Provide key and secrets) let twitter = Twitter ( UserContext ( ctx )) // Search for the phrases and start the stream let search = twitter . Streaming . FilterTweets phrases search . Start ()

The search.TweetReceived event will be triggered when a new tweet happens. The status object has a bunch of properties (inferred by a type provider). It turns out that event status.Text is optional and so parsing the tweets involves a lot of pattern matching:

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: let liveTweets = search . TweetReceived |> Observable . choose ( fun status -> // Parse the location, if the tweet has it let origLocation = parseLocation status . Geo // Get user name, text of the tweet and location match status . User , status . Text with | Some user , Some text -> { Tweeted = DateTime . UtcNow ; OriginalArea = user . Location Text = text ; PictureUrl = user . ProfileImageUrl ; (Populate other properties) } |> Some | _ -> None )

The code is slightly simplified, but it is pretty representative. Now we have a value liveTweets of type IObservable<Tweet> which is an event that is triggered every time we get a new (not completely silly) tweet.

Geolocating tweets and users

The hardest bit turns out to be getting good tweets for the map. Not a lot of tweets come with GPS coordinates and so I had to do a couple of tricks. When more people start tweeting around the New Year, we should be able to use mostly tweets with GPS coordinates, but there are some backup strategies:

If a tweet has GPS coordinates, use this as the location Every now and then use MapQuest or Bing to geolocate the user based on their location in the profile If we didn't produce enough tweets using (1) or (2), locate tweet based on the language of the phrase and put it in some place where a previous tweet with the same phrase appeared.

In priciple, geolocating users based on their profile would work good enough, but all the geolocation services have rate limits that are easy to hit when the site is running live and so I added (3) as the last resort. If I had more time, I would probably try to build an index with country and city names, which would likely cover enough tweets (at least from users with a reasonable text in their "location").

Tweets with GPS coordinates

All of the methods report tweets to a "replay" agent (see below) that replays the tweets with a specified delay. This is done using replay.AddEvent at the end of the pipeline. For tweets with GPS coordinates, we simply copy the already provided data to InferredLocation (coordinates) and InferredArea (text):

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: liveTweets |> Observable . choose ( fun tw -> match tw . OriginalLocation with | Some loc -> ( tw . Tweeted . AddSeconds ( 5.0 ), { tw with InferredArea = Some tw . OriginalArea InferredLocation = Some loc }) |> Some | _ -> None ) |> Observable . add replay . AddEvent

Geolocating tweets using Bing and MapQuest

For locating tweets based on the user's location, we will be calling Bing and MapQuest APIs. This is done using type providers (see below) and wrapped in a nice MapQuest.locate and Bing.locate functions. We also need to limit rate at which we use these - the following geolocates one tweet per 5 seconds using MapQuest:

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: liveTweets |> Observable . limitRate 5000 |> Observable . mapAsyncIgnoreErrors ( fun tw -> async { let! located = MapQuest . locate tw . OriginalArea return located |> Option . map ( fun ( area , loc ) -> tw . Tweeted . AddSeconds ( 10.0 ), { tw with InferredLocation = Some loc ; InferredArea = Some area }) }) |> Observble . choose id |> Observable . add replay . AddEvent

Time zones and geolocating with type providers

As in every F# project, I'm making a heavy use of F# Data type providers when calling REST-based geolocation services. As a bonus, I also needed to find time zones of countries of the world, which can be done by extracting the information from List of time zones by country Wikipedia page using the HTML type provider.

Extracting time zone information

The HTML type provider gives us access to the tables on the Wikipedia page and so we can get the country and time zones just by writing r.Country and r.''Time Zone'' (using backticks to wrap the space). As far as I know, Datamaps does not easily let me display multiple time zones per country and so I just pick the middle time zone:

1: 2: 3: 4: 5: 6: 7: 8: 9: type TimeZones = HtmlProvider < "http://.../List_of_time_zones_by_country" > let reg = Regex ( """UTC([\+\-][0-9][0-9]\:[0-9][0-9])?""" ) let timeZones = [ for r in TimeZones . GetSample () . Tables . Table1 . Rows do let tz = r . ``Time Zone`` . Replace ( "−" , "-" ) let matches = reg . Matches ( tz ) if matches . Count > 0 then yield r . Country , matches . [ matches . Count / 2 ] . Value ]

There are a few explicitly defined countries in the actual source code for countries where the middle time zone is very wrong and for countries that are named differently on Wikipedia.

Geolocating using MapQuest

Both Bing and MapQuest provide a nice REST end-point that we can call using the JSON type provider. To compose the sample URL, we need to use the Literal attribute and append a key (which is stored in a separate config file). The JSON type provider infers the type from the response and gives us nice typed access to the results:

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: // Use JSON provider to get a type for calling the API let [< Literal >] MapQuestSample = "http://mapquestapi.com/geocoding/v1/address?location=Prague" type MapQuest = JsonProvider < MapQuestSample > let locate ( place : string ) = let url = "http://www.mapquestapi.com/geocoding/v1/address?key=" + Config . MapQuestKey + "&location=" + ( HttpUtility . UrlEncode place ) MapQuest . Load ( url ) . Results |> Seq . choose ( fun loc -> // Pick the first returned location if there were any if loc . Locations . Length = 0 then None else Some ( loc , loc . Locations . [ 0 ]) ) |> Seq . map ( fun ( info , loc ) -> // Return the location with lattitude and longitude info . ProvidedLocation . Location , ( loc . LatLng . Lat , loc . LatLng . Lng ) )

As usual, using the JSON type provider for calling REST APIs makes things very easy. The Results property is inferred to be an array of records and information such as loc.LatLng.Lat is also statically typed.

Reactive programming with F# agents

The project does quite a lot of interesting reactive event processing. In F#, you can, of course, use Reactive Extensions (Rx), but I always found Rx a bit hard to use because they lack simple underlying primitives (more about this in my rant on library design). F# comes with a simple set of primitives in the Observable module which covers some 80% of what you need and you can easily implement additional primitives using F# agents.

For example, the following is a simple agent that I wrote to limit the rate of requests. The idea is that the agent will emit an event it receives and then it will ignore all other events for the specified number of milliseconds:

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: /// Limits the rate of emitted messages to at most /// one per the specified number of milliseconds type RateLimitAgent < ' T > ( timeout ) = let event = Event < ' T > () let agent = MailboxProcessor . Start ( fun inbox -> // We remember the last time we emitted a message let rec loop ( lastMessageTime : DateTime ) = async { let! e = inbox . Receive () let now = DateTime . UtcNow // If we waited long enough, report the event // otherwise ignore it and wait some more let ms = ( now - lastMessageTime ) . TotalMilliseconds if ms > timeout then event . Trigger ( e ) return! loop now else return! loop lastMessageTime } loop DateTime . MinValue ) /// Triggered when an event happens member x . EventOccurred = event . Publish /// Send an event to the agent member x . AddEvent ( event ) = agent . Post ( event )

Agents are the much needed lower level primitive of the Reactive Extensions. You can quite easily express any logic you need using just a state machine encoded as a recursive asynchronous loop. The implementation then wraps the agent in a higher-level primitive Observable.limitRate that was used in the earlier snippet.

Handling websockets with Suave

One more nice thing in the project is the handling of web sockets. The server serves static files from the web sub-directory, but it also communicates with the front-end via three web sockets (for the map, feed and wordcloud). When a client connects, we simply want to start sending updates to it from one of the IObservable<T> events that we defined earlier (e.g. by serializing tweets from liveTweets as JSON).

To do this, I first defined a helper socketOfObservable , which uses Suave's socket { .. } computation builder and repeatedly awaits an update from the specified updates and reports it to via the socket:

1: 2: 3: 4: 5: 6: 7: 8: let socketOfObservable updates ( webSocket : WebSocket ) ctx = socket { while true do // Wait for the next update from the source let! update = updates |> Async . AwaitObservable |> SocketOp . ofAsync // Report it to the front-end over the wire! do! webSocket . send Text ( UTF8 . bytes update ) true }

The main server is then composed from a number of web parts - the first three handle the communication via web sockets, the fourth one returns information about time zones that we downloaded from Wikipedia and the last two serve static files:

1: 2: 3: 4: 5: 6: 7: 8: let part = choose [ path "/maptweets" > > = handShake ( socketOfObservable mapTweets ) path "/feedtweets" > > = handShake ( socketOfObservable feedTweets ) path "/frequencies" > > = handShake ( socketOfObservable phraseUpdates ) path "/zones" > > = Successful . OK timeZonesJson path "/" > > = Files . browseFile root "index.html" Files . browse root ]

Summary

The main part of the project in app.fsx is some 350 lines long and I find it pretty amazing how much you can do in this small number of lines. If you're writing a project like this in F#, you get to use a number of nice libraries including F# Data Toolbox for the Twitter API, Suave.io for the web server and F# Data type providers for calling REST APIs. Finally, I deployed the service using Azure VM, but you could also use MBrace which can host web servers in a cluster, or any other hosting - all the libraries I'm using are cross-platform.

If you're reading this around December 31, 2015 then definitely check out the project running live. I didn't plan to turn this into a reusable application, but who knows! :-) If you want to use it for tracking tweets related to some other events you can find the full source on GitHub under the Apache license - and also get in touch if you have some interesting use for this work!

namespace System

namespace System.Web

namespace System.Collections

namespace System.Collections.Generic

Multiple items

namespace FSharp



--------------------

namespace Microsoft.FSharp

Multiple items

namespace FSharp.Data



--------------------

namespace Microsoft.FSharp.Data

namespace FSharp.Data.Toolbox

namespace FSharp.Data.Toolbox.Twitter

module AsyncHelpers

namespace System.Text

namespace System.Text.RegularExpressions

namespace Suave

module Web



from Suave

module Http



from Suave

module Applicatives



from Suave.Http

namespace Suave.Sockets

namespace Suave.Sockets.Control

module AsyncSocket



from Suave.Sockets

module WebSocket



from Suave

namespace Suave.Utils

type Tweet =

{Tweeted: DateTime;

Text: string;

OriginalArea: string;

UserName: string;

UserScreenName: string;

PictureUrl: string;

OriginalLocation: (decimal * decimal) option;

Phrase: int;

IsRetweet: bool;

GeoLocationSource: string;

...}



Full name: Happy-new-year-tweets.Tweet





Information we collect about tweets. The `Inferred` fields are calculated later

by geolocating the user, all other information is filled when tweet is received

Tweet.Tweeted: DateTime

Multiple items

type DateTime =

struct

new : ticks:int64 -> DateTime + 10 overloads

member Add : value:TimeSpan -> DateTime

member AddDays : value:float -> DateTime

member AddHours : value:float -> DateTime

member AddMilliseconds : value:float -> DateTime

member AddMinutes : value:float -> DateTime

member AddMonths : months:int -> DateTime

member AddSeconds : value:float -> DateTime

member AddTicks : value:int64 -> DateTime

member AddYears : value:int -> DateTime

...

end



Full name: System.DateTime



--------------------

DateTime()

(+0 other overloads)

DateTime(ticks: int64) : unit

(+0 other overloads)

DateTime(ticks: int64, kind: DateTimeKind) : unit

(+0 other overloads)

DateTime(year: int, month: int, day: int) : unit

(+0 other overloads)

DateTime(year: int, month: int, day: int, calendar: Globalization.Calendar) : unit

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int) : unit

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int, kind: DateTimeKind) : unit

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int, calendar: Globalization.Calendar) : unit

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int, millisecond: int) : unit

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int, millisecond: int, kind: DateTimeKind) : unit

(+0 other overloads)

Multiple items

Tweet.Text: string



--------------------

namespace System.Text

Multiple items

val string : value:'T -> string



Full name: Microsoft.FSharp.Core.Operators.string



--------------------

type string = String



Full name: Microsoft.FSharp.Core.string

Tweet.OriginalArea: string

Tweet.UserName: string

Tweet.UserScreenName: string

Tweet.PictureUrl: string

Tweet.OriginalLocation: (decimal * decimal) option

type 'T option = Option<'T>



Full name: Microsoft.FSharp.Core.option<_>

Multiple items

val decimal : value:'T -> decimal (requires member op_Explicit)



Full name: Microsoft.FSharp.Core.Operators.decimal



--------------------

type decimal = Decimal



Full name: Microsoft.FSharp.Core.decimal



--------------------

type decimal<'Measure> = decimal



Full name: Microsoft.FSharp.Core.decimal<_>

Tweet.Phrase: int

Multiple items

val int : value:'T -> int (requires member op_Explicit)



Full name: Microsoft.FSharp.Core.Operators.int



--------------------

type int = int32



Full name: Microsoft.FSharp.Core.int



--------------------

type int<'Measure> = int



Full name: Microsoft.FSharp.Core.int<_>

Tweet.IsRetweet: bool

type bool = Boolean



Full name: Microsoft.FSharp.Core.bool

Tweet.GeoLocationSource: string

Tweet.InferredArea: string option

Tweet.InferredLocation: (decimal * decimal) option

val root : string



Full name: Happy-new-year-tweets.root

namespace System.IO

type Path =

static val DirectorySeparatorChar : char

static val AltDirectorySeparatorChar : char

static val VolumeSeparatorChar : char

static val InvalidPathChars : char[]

static val PathSeparator : char

static member ChangeExtension : path:string * extension:string -> string

static member Combine : [<ParamArray>] paths:string[] -> string + 3 overloads

static member GetDirectoryName : path:string -> string

static member GetExtension : path:string -> string

static member GetFileName : path:string -> string

...



Full name: System.IO.Path

IO.Path.Combine([<ParamArray>] paths: string []) : string

IO.Path.Combine(path1: string, path2: string) : string

IO.Path.Combine(path1: string, path2: string, path3: string) : string

IO.Path.Combine(path1: string, path2: string, path3: string, path4: string) : string

val phrases : string list



Full name: Happy-new-year-tweets.phrases

"manigong bagong taon"; "Срећна Нова година"; "честита нова година"; "selamat tahun baru";

"С Новым Годом"; "あけまして おめでとう ございます"; "新年快乐"; "Щасливого Нового Року"; "שנה טובה"

"yeni yılınız kutlu olsun"; "feliz año nuevo"; "happy new year"; "Καλή Χρονιά";"godt nyttår"

"bon any nou"; "felice anno nuovo"; "sretna nova godina"; "godt nytår"; "gelukkig nieuwjaar"

"Frohes neues Jahr"; "urte berri on"; "bonne année"; "boldog új évet"; "gott nytt år"

"szczęśliwego nowego roku"; "blwyddyn newydd dda"; "feliz ano novo"; "sugeng warsa enggal"

val ctx : TwitterUserContext



Full name: Happy-new-year-tweets.ctx

{ ConsumerKey = Config.TwitterKey; ConsumerSecret = Config.TwitterSecret;

AccessToken = Config.TwitterAccessToken; AccessSecret = Config.TwitterAccessSecret }

val twitter : Twitter



Full name: Happy-new-year-tweets.twitter

Multiple items

type Twitter =

new : context:TwitterContext -> Twitter

member RequestRawData : url:string * query:(string * string) list -> string

member Connections : Connections

member Search : Search

member Streaming : Streaming

member Timelines : Timelines

member Users : Users

static member Authenticate : consumer_key:string * consumer_secret:string -> TwitterConnector

static member AuthenticateAppOnly : consumer_key:string * consumer_secret:string -> Twitter

static member TwitterWeb : unit -> WebBrowser



Full name: FSharp.Data.Toolbox.Twitter.Twitter



--------------------

new : context:TwitterContext -> Twitter

union case TwitterContext.UserContext: TwitterUserContext -> TwitterContext

val search : TwitterStream<JsonProvider<...>.Root>



Full name: Happy-new-year-tweets.search

property Twitter.Streaming: Streaming

member Streaming.FilterTweets : keywords:seq<string> -> TwitterStream<JsonProvider<...>.Root>

abstract member TwitterStream.Start : unit -> unit

val liveTweets : IObservable<Tweet>



Full name: Happy-new-year-tweets.liveTweets

property TwitterStream.TweetReceived: IEvent<JsonProvider<...>.Root>

Multiple items

module Observable



from AsyncHelpers



--------------------

module Observable



from Microsoft.FSharp.Control

val choose : chooser:('T -> 'U option) -> source:IObservable<'T> -> IObservable<'U>



Full name: Microsoft.FSharp.Control.Observable.choose

val status : JsonProvider<...>.Root

val origLocation : (decimal * decimal) option

property JsonProvider<...>.Root.Geo: Option<JsonProvider<...>.Geo>

property JsonProvider<...>.Root.User: Option<JsonProvider<...>.User>

property JsonProvider<...>.Root.Text: Option<string>

union case Option.Some: Value: 'T -> Option<'T>

val user : JsonProvider<...>.User

val text : string

property DateTime.UtcNow: DateTime

Multiple items

union case Opcode.Text: Opcode



--------------------

namespace System.Text

UserName = user.Name; UserScreenName = user.ScreenName;

OriginalLocation = origLocation; Phrase = getPhrase text

InferredArea = None; InferredLocation = None;

IsRetweet = isRT status; GeoLocationSource = "NA"

union case Option.None: Option<'T>

val tw : Tweet

val loc : decimal * decimal

DateTime.AddSeconds(value: float) : DateTime

val add : callback:('T -> unit) -> source:IObservable<'T> -> unit



Full name: Microsoft.FSharp.Control.Observable.add

union case AlternativeSourceAgentMessage.AddEvent: 'T -> AlternativeSourceAgentMessage<'T>

val limitRate : milliseconds:int -> source:IObservable<'a> -> IObservable<'a>



Full name: AsyncHelpers.Observable.limitRate





Limits the rate of emitted messages to at most one per the specified number of milliseconds

val mapAsyncIgnoreErrors : f:('a -> Async<'b>) -> source:IObservable<'a> -> IObservable<'b>



Full name: AsyncHelpers.Observable.mapAsyncIgnoreErrors





Behaves like `Observable.map`, but does not stop when error happens

val async : AsyncBuilder



Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.async

val located : (string * (decimal * decimal)) option

Multiple items

module Option



from Suave.Utils



--------------------

module Option



from Microsoft.FSharp.Core

val map : mapping:('T -> 'U) -> option:'T option -> 'U option



Full name: Microsoft.FSharp.Core.Option.map

val area : string

val choose : options:Types.WebPart list -> Types.WebPart



Full name: Suave.Http.choose

val id : x:'T -> 'T



Full name: Microsoft.FSharp.Core.Operators.id

type TimeZones = HtmlProvider<...>



Full name: Happy-new-year-tweets.TimeZones

type HtmlProvider



Full name: FSharp.Data.HtmlProvider





<summary>Typed representation of an HTML file.</summary>

<param name='Sample'>Location of an HTML sample file or a string containing a sample HTML document.</param>

<param name='PreferOptionals'>When set to true, inference will prefer to use the option type instead of nullable types, `double.NaN` or `""` for missing values. Defaults to false.</param>

<param name='IncludeLayoutTables'>Includes tables that are potentially layout tables (with cellpadding=0 and cellspacing=0 attributes)</param>

<param name='MissingValues'>The set of strings recogized as missing values. Defaults to `NaN,NA,#N/A,:,-,TBA,TBD`.</param>

<param name='Culture'>The culture used for parsing numbers and dates. Defaults to the invariant culture.</param>

<param name='Encoding'>The encoding used to read the sample. You can specify either the character set name or the codepage number. Defaults to UTF8 for files, and to ISO-8859-1 the for HTTP requests, unless `charset` is specified in the `Content-Type` response header.</param>

<param name='ResolutionFolder'>A directory that is used when resolving relative file references (at design time and in hosted execution).</param>

<param name='EmbeddedResource'>When specified, the type provider first attempts to load the sample from the specified resource

(e.g. 'MyCompany.MyAssembly, resource_name.html'). This is useful when exposing types generated by the type provider.</param>

"https://en.wikipedia.org/wiki/List_of_time_zones_by_country"

val reg : Regex



Full name: Happy-new-year-tweets.reg

Multiple items

type Regex =

new : pattern:string -> Regex + 1 overload

member GetGroupNames : unit -> string[]

member GetGroupNumbers : unit -> int[]

member GroupNameFromNumber : i:int -> string

member GroupNumberFromName : name:string -> int

member IsMatch : input:string -> bool + 1 overload

member Match : input:string -> Match + 2 overloads

member Matches : input:string -> MatchCollection + 1 overload

member Options : RegexOptions

member Replace : input:string * replacement:string -> string + 5 overloads

...



Full name: System.Text.RegularExpressions.Regex



--------------------

Regex(pattern: string) : unit

Regex(pattern: string, options: RegexOptions) : unit

val timeZones : (string * string) list



Full name: Happy-new-year-tweets.timeZones

val r : HtmlProvider<...>.Table1.Row

HtmlProvider<...>.GetSample() : HtmlProvider<...>

val tz : string

val matches : MatchCollection

Regex.Matches(input: string) : MatchCollection

Regex.Matches(input: string, startat: int) : MatchCollection

property MatchCollection.Count: int

property HtmlProvider<...>.Table1.Row.Country: string

Multiple items

type LiteralAttribute =

inherit Attribute

new : unit -> LiteralAttribute



Full name: Microsoft.FSharp.Core.LiteralAttribute



--------------------

new : unit -> LiteralAttribute

val MapQuestSample : string



Full name: Happy-new-year-tweets.MapQuestSample

"http://www.mapquestapi.com/geocoding/v1/address?location=Prague&key=" + Config.MapQuestKey

type MapQuest = JsonProvider<...>



Full name: Happy-new-year-tweets.MapQuest

type JsonProvider



Full name: FSharp.Data.JsonProvider





<summary>Typed representation of a JSON document.</summary>

<param name='Sample'>Location of a JSON sample file or a string containing a sample JSON document.</param>

<param name='SampleIsList'>If true, sample should be a list of individual samples for the inference.</param>

<param name='RootName'>The name to be used to the root type. Defaults to `Root`.</param>

<param name='Culture'>The culture used for parsing numbers and dates. Defaults to the invariant culture.</param>

<param name='Encoding'>The encoding used to read the sample. You can specify either the character set name or the codepage number. Defaults to UTF8 for files, and to ISO-8859-1 the for HTTP requests, unless `charset` is specified in the `Content-Type` response header.</param>

<param name='ResolutionFolder'>A directory that is used when resolving relative file references (at design time and in hosted execution).</param>

<param name='EmbeddedResource'>When specified, the type provider first attempts to load the sample from the specified resource

(e.g. 'MyCompany.MyAssembly, resource_name.json'). This is useful when exposing types generated by the type provider.</param>

<param name='InferTypesFromValues'>If true, turns on additional type inference from values.

(e.g. type inference infers string values such as "123" as ints and values constrained to 0 and 1 as booleans.)</param>

val locate : place:string -> seq<string * (decimal * decimal)>



Full name: Happy-new-year-tweets.locate

val place : string

val url : string

module Config

val MapQuestKey : string



Full name: Config.MapQuestKey

Multiple items

type HttpUtility =

new : unit -> HttpUtility

static member HtmlAttributeEncode : s:string -> string + 1 overload

static member HtmlDecode : s:string -> string + 1 overload

static member HtmlEncode : s:string -> string + 2 overloads

static member JavaScriptStringEncode : value:string -> string + 1 overload

static member ParseQueryString : query:string -> NameValueCollection + 1 overload

static member UrlDecode : str:string -> string + 3 overloads

static member UrlDecodeToBytes : str:string -> byte[] + 3 overloads

static member UrlEncode : str:string -> string + 3 overloads

static member UrlEncodeToBytes : str:string -> byte[] + 3 overloads

...



Full name: System.Web.HttpUtility



--------------------

HttpUtility() : unit

HttpUtility.UrlEncode(bytes: byte []) : string

HttpUtility.UrlEncode(str: string) : string

HttpUtility.UrlEncode(str: string, e: Encoding) : string

HttpUtility.UrlEncode(bytes: byte [], offset: int, count: int) : string

JsonProvider<...>.Load(uri: string) : JsonProvider<...>.Root





Loads JSON from the specified uri

JsonProvider<...>.Load(reader: IO.TextReader) : JsonProvider<...>.Root





Loads JSON from the specified reader

JsonProvider<...>.Load(stream: IO.Stream) : JsonProvider<...>.Root





Loads JSON from the specified stream

module Seq



from Microsoft.FSharp.Collections

val choose : chooser:('T -> 'U option) -> source:seq<'T> -> seq<'U>



Full name: Microsoft.FSharp.Collections.Seq.choose

val loc : JsonProvider<...>.Result

property JsonProvider<...>.Result.Locations: JsonProvider<...>.Location []

property Array.Length: int

val map : mapping:('T -> 'U) -> source:seq<'T> -> seq<'U>



Full name: Microsoft.FSharp.Collections.Seq.map

val info : JsonProvider<...>.Result

val loc : JsonProvider<...>.Location

property JsonProvider<...>.Result.ProvidedLocation: JsonProvider<...>.ProvidedLocation

property JsonProvider<...>.ProvidedLocation.Location: string

property JsonProvider<...>.Location.LatLng: JsonProvider<...>.LatLng

property JsonProvider<...>.LatLng.Lat: decimal

property JsonProvider<...>.LatLng.Lng: decimal

Multiple items

type RateLimitAgent<'T> =

new : timeout:float -> RateLimitAgent<'T>

member AddEvent : event:'T -> unit

member EventOccurred : IEvent<'T>



Full name: Happy-new-year-tweets.RateLimitAgent<_>





Limits the rate of emitted messages to at most

one per the specified number of milliseconds



--------------------

new : timeout:float -> RateLimitAgent<'T>

val timeout : float

val event : Event<'T>

Multiple items

module Event



from Microsoft.FSharp.Control



--------------------

type Event<'T> =

new : unit -> Event<'T>

member Trigger : arg:'T -> unit

member Publish : IEvent<'T>



Full name: Microsoft.FSharp.Control.Event<_>



--------------------

type Event<'Delegate,'Args (requires delegate and 'Delegate :> Delegate)> =

new : unit -> Event<'Delegate,'Args>

member Trigger : sender:obj * args:'Args -> unit

member Publish : IEvent<'Delegate,'Args>



Full name: Microsoft.FSharp.Control.Event<_,_>



--------------------

new : unit -> Event<'T>



--------------------

new : unit -> Event<'Delegate,'Args>

val agent : MailboxProcessor<'T>

Multiple items

type MailboxProcessor<'Msg> =

interface IDisposable

new : body:(MailboxProcessor<'Msg> -> Async<unit>) * ?cancellationToken:CancellationToken -> MailboxProcessor<'Msg>

member Post : message:'Msg -> unit

member PostAndAsyncReply : buildMessage:(AsyncReplyChannel<'Reply> -> 'Msg) * ?timeout:int -> Async<'Reply>

member PostAndReply : buildMessage:(AsyncReplyChannel<'Reply> -> 'Msg) * ?timeout:int -> 'Reply

member PostAndTryAsyncReply : buildMessage:(AsyncReplyChannel<'Reply> -> 'Msg) * ?timeout:int -> Async<'Reply option>

member Receive : ?timeout:int -> Async<'Msg>

member Scan : scanner:('Msg -> Async<'T> option) * ?timeout:int -> Async<'T>

member Start : unit -> unit

member TryPostAndReply : buildMessage:(AsyncReplyChannel<'Reply> -> 'Msg) * ?timeout:int -> 'Reply option

...



Full name: Microsoft.FSharp.Control.MailboxProcessor<_>



--------------------

new : body:(MailboxProcessor<'Msg> -> Async<unit>) * ?cancellationToken:Threading.CancellationToken -> MailboxProcessor<'Msg>

static member MailboxProcessor.Start : body:(MailboxProcessor<'Msg> -> Async<unit>) * ?cancellationToken:Threading.CancellationToken -> MailboxProcessor<'Msg>

val inbox : MailboxProcessor<'T>

val loop : (DateTime -> Async<'a>)

val lastMessageTime : DateTime

val e : 'T

member MailboxProcessor.Receive : ?timeout:int -> Async<'Msg>

val now : DateTime

val ms : float

member Event.Trigger : arg:'T -> unit

field DateTime.MinValue

val x : RateLimitAgent<'T>

member RateLimitAgent.EventOccurred : IEvent<'T>



Full name: Happy-new-year-tweets.RateLimitAgent`1.EventOccurred





Triggered when an event happens

property Event.Publish: IEvent<'T>

member RateLimitAgent.AddEvent : event:'T -> unit



Full name: Happy-new-year-tweets.RateLimitAgent`1.AddEvent





Send an event to the agent

val event : 'T

member MailboxProcessor.Post : message:'Msg -> unit

val socketOfObservable : updates:IObservable<string> -> webSocket:WebSocket -> ctx:'a -> Async<Choice<unit,Error>>



Full name: Happy-new-year-tweets.socketOfObservable

val updates : IObservable<string>

val webSocket : WebSocket

Multiple items

module WebSocket



from Suave



--------------------

type WebSocket =

new : connection:Connection -> WebSocket

member read : unit -> Async<Choice<(Opcode * byte [] * bool),Error>>

member send : opcode:Opcode -> bs:byte [] -> fin:bool -> Async<Choice<unit,Error>>



Full name: Suave.WebSocket.WebSocket



--------------------

new : connection:Connection -> WebSocket

val ctx : 'a

val socket : SocketMonad



Full name: Suave.Sockets.Control.SocketMonad.socket

val update : string

Multiple items

module Async



from Suave.Utils



--------------------

type Async

static member AsBeginEnd : computation:('Arg -> Async<'T>) -> ('Arg * AsyncCallback * obj -> IAsyncResult) * (IAsyncResult -> 'T) * (IAsyncResult -> unit)

static member AwaitEvent : event:IEvent<'Del,'T> * ?cancelAction:(unit -> unit) -> Async<'T> (requires delegate and 'Del :> Delegate)

static member AwaitIAsyncResult : iar:IAsyncResult * ?millisecondsTimeout:int -> Async<bool>

static member AwaitTask : task:Task -> Async<unit>

static member AwaitTask : task:Task<'T> -> Async<'T>

static member AwaitWaitHandle : waitHandle:WaitHandle * ?millisecondsTimeout:int -> Async<bool>

static member CancelDefaultToken : unit -> unit

static member Catch : computation:Async<'T> -> Async<Choice<'T,exn>>

static member FromBeginEnd : beginAction:(AsyncCallback * obj -> IAsyncResult) * endAction:(IAsyncResult -> 'T) * ?cancelAction:(unit -> unit) -> Async<'T>

static member FromBeginEnd : arg:'Arg1 * beginAction:('Arg1 * AsyncCallback * obj -> IAsyncResult) * endAction:(IAsyncResult -> 'T) * ?cancelAction:(unit -> unit) -> Async<'T>

static member FromBeginEnd : arg1:'Arg1 * arg2:'Arg2 * beginAction:('Arg1 * 'Arg2 * AsyncCallback * obj -> IAsyncResult) * endAction:(IAsyncResult -> 'T) * ?cancelAction:(unit -> unit) -> Async<'T>

static member FromBeginEnd : arg1:'Arg1 * arg2:'Arg2 * arg3:'Arg3 * beginAction:('Arg1 * 'Arg2 * 'Arg3 * AsyncCallback * obj -> IAsyncResult) * endAction:(IAsyncResult -> 'T) * ?cancelAction:(unit -> unit) -> Async<'T>

static member FromContinuations : callback:(('T -> unit) * (exn -> unit) * (OperationCanceledException -> unit) -> unit) -> Async<'T>

static member Ignore : computation:Async<'T> -> Async<unit>

static member OnCancel : interruption:(unit -> unit) -> Async<IDisposable>

static member Parallel : computations:seq<Async<'T>> -> Async<'T []>

static member RunSynchronously : computation:Async<'T> * ?timeout:int * ?cancellationToken:CancellationToken -> 'T

static member Sleep : millisecondsDueTime:int -> Async<unit>

static member Start : computation:Async<unit> * ?cancellationToken:CancellationToken -> unit

static member StartAsTask : computation:Async<'T> * ?taskCreationOptions:TaskCreationOptions * ?cancellationToken:CancellationToken -> Task<'T>

static member StartChild : computation:Async<'T> * ?millisecondsTimeout:int -> Async<Async<'T>>

static member StartChildAsTask : computation:Async<'T> * ?taskCreationOptions:TaskCreationOptions -> Async<Task<'T>>

static member StartImmediate : computation:Async<unit> * ?cancellationToken:CancellationToken -> unit

static member StartWithContinuations : computation:Async<'T> * continuation:('T -> unit) * exceptionContinuation:(exn -> unit) * cancellationContinuation:(OperationCanceledException -> unit) * ?cancellationToken:CancellationToken -> unit

static member SwitchToContext : syncContext:SynchronizationContext -> Async<unit>

static member SwitchToNewThread : unit -> Async<unit>

static member SwitchToThreadPool : unit -> Async<unit>

static member TryCancelled : computation:Async<'T> * compensation:(OperationCanceledException -> unit) -> Async<'T>

static member CancellationToken : Async<CancellationToken>

static member DefaultCancellationToken : CancellationToken



Full name: Microsoft.FSharp.Control.Async



--------------------

type Async<'T>



Full name: Microsoft.FSharp.Control.Async<_>

static member Async.AwaitObservable : ev1:IObservable<'T1> -> Async<'T1>





Creates an asynchronous workflow that will be resumed when the

specified observables produces a value. The workflow will return

the value produced by the observable.

Multiple items

module SocketOp



from Suave.Sockets



--------------------

type SocketOp<'a> = Async<Choice<'a,Error>>



Full name: Suave.Sockets.SocketOp<_>

val ofAsync : a:Async<'a> -> SocketOp<'a>



Full name: Suave.Sockets.SocketOp.ofAsync

member WebSocket.send : opcode:Opcode -> bs:byte [] -> fin:bool -> Async<Choice<unit,Error>>

module UTF8



from Suave.Utils

val bytes : s:string -> byte []



Full name: Suave.Utils.UTF8.bytes

val part : Types.WebPart



Full name: Happy-new-year-tweets.part

val path : s:string -> Types.WebPart



Full name: Suave.Http.Applicatives.path

val handShake : continuation:(WebSocket -> Types.HttpContext -> SocketOp<unit>) -> ctx:Types.HttpContext -> Async<Types.HttpContext option>



Full name: Suave.WebSocket.handShake

module Successful



from Suave.Http

val OK : a:string -> Types.WebPart



Full name: Suave.Http.Successful.OK

module Files



from Suave.Http

val browseFile : rootPath:string -> fileName:string -> Types.WebPart



Full name: Suave.Http.Files.browseFile

val browse : rootPath:string -> Types.WebPart



Full name: Suave.Http.Files.browse