Building Twitter with Apigee App Services

Twitter seems like a very simple app on the surface. 140 characters, users following users - building an app like that should be easy, right?  It is until you start to look under the hood.  A quick examination reveals that there are a number of fairly complex features that need to be implemented on the backend: user management, activity streams, support for connections (users following users), and all of this packaged up in a nice, easy-to-use API.

In this article, we look at how we used Apigee App Services to quickly build out our Twitter clone called Messagee using JavaScript, HTML5, and our JavaScript SDK. The source code and working demo are available on Github: Source Code | Working Demo

Messagee

The basic functionality of our app is, as with Twitter, users can post 140 character messages, or “tweets” to the system.  Users can also become “followers” of other users.  When users log in to check their messages, they can see either a general feed of all messages posted to the system or a personalized message “feed”. This personalized feed consists of messages the user has posted as well as messages from users they follow.

Image: FreeDigitalPhotos.net

Getting Started

The starting point for building HTML5 based apps with the App Services API is the JavaScript SDK.  Download it, then include the file at the top of your code:

<script src="js/usergrid.js" type="text/javascript" />

Next, create a new client to use to make your calls. Make sure you specify your Organization and Application names. (Create these in the App Services Admin Portal.)

  var client = new Usergrid.Client({
    orgName:'ApigeeOrg', //your orgname goes here (not case sensitive)
    appName:'MessageeApp', //your appname goes here (not case sensitive)
    logging: true, //optional - turn on logging, off by default
    buildCurl: true //optional - turn on curl commands, off by default
  });

The JavaScript SDK makes it easy to work with the API by providing the infrastructure that makes the calls to the server, including encoding and decoding any JSON.

User Management

App builders often start by thinking about user management. In the case of our Twitter clone, every user will need their own account. The most basic need is to track usernames and passwords so that users can log into their accounts.  But we also need to track other user information such as their picture, email, and the users they are following.

This functionality is all built into App Services, so the only work we need to do in our app is to build the forms to support these features - a new user creation form, a form to update a user’s account settings, and a login form.  In each case, instead of submitting the form back to the server, we simply attach the submit button to a click event handler.  That handler calls the appropriate function to grab the form data and take the appropriate action.

New user creation form

The new user creation form, located in the index.html file, simply asks for standard user information.  We use a similar form/function to handle updating a user’s account settings.

<form name="form-create-new-account" id="form-create-new-account">
<label for="new-name">Name</label>
<input type="text" name="new-name" id="new-name" class="span4" />
<label for="new-email">Email</label>
<input type="text" name="new-email" id="new-email" class="span4" />
<label for="new-username">Username</label>
<input type="text" name="new-username" id="new-username" class="span4" />
<label for="new-password">Password</label>
<input type="password" name="new-password" id="new-password" class="span4" />
</form>
<div style="width: 50%; float: left">
<button id="btn-create-new-account">Go!</button>
</div>

When the user clicks the submit button (btn-create-new-account), the click event is caught by the following handler:

  $('#btn-create-new-account').bind('click', createNewUser);

The click event handler then calls the createNewUser function, which, after validating the form input, creates a new Entity object of type user and saves it back to the API.  If the call to save the user is successful, the login form is populated with the username and password, and the user is automatically logged in.  On failure, the user is returned to the main login screen and shown an error.

  var options = {
    type:'users',
    username:username,
    password:password,
    name:name,
    email:email
  }

  client.createEntity(options, function (err, newUser) {
    if (err){
       window.location = "#login";
      $('#login-section-error').html('There was an error creating the new user.');
    } else {
      appUser = newUser;
      //new user is created, so set their values in the login form and call login
      $("#username").val(username);
      $("#password").val(password);
      login();
    }
  });

Login form

The login form takes only a username and password as form parameters.  

<form name="form-login" id="form-login">
<label for="username">Username</label>
<input type="text" name="username" id="username" class="span4" />
<label for="password">Password</label>
<input type="password" name="password" id="password" class="span4" />
</form>
<div style="width: 50%; float: left">
<a href="#login" id="btn-login" data-role="button">Login</a>
</div>

Again, the submit button (btn-login) click event is caught by an event handler which in turn calls the login() function.

  $('#btn-login').bind('click', login);

The login function merely grabs the username and password from the form and attempts to log the user in by calling the client.logIn function from the SDK. If the call is successful and the user is logged in, they are taken to the main page of the app: the messages list page.

  $('#login-section-error').html('');
  var username = $("#username").val();
  var password = $("#password").val();

  client.login(username, password,
    function (err) {
      if (err) {
        $('#login-section-error').html('There was an error logging you in.');
      } else {
        //login succeeded
        client.getLoggedInUser(function(err, data, user) {
          if(err) {
            //error - could not get logged in user
          } else {
            if (client.isLoggedIn()){
              appUser = user;
             // showFullFeed();
            }
          }
        });

        //clear out the login form so it is empty if the user chooses to log out
        $("#username").val('');
        $("#password").val('');

        //default to the full feed view (all messages in the system)
        showMyFeed();
      }
    }
  );

Apigee App Services makes it easy to manage users since all the necessary back-end infrastructure is already built and ready to go.


Activity Streams

Activity Streams can be used to turn individual tweets into a proper user feed, bridging the gap between a simple message and an interactive social messaging application. Here we turn our attention towards the the most important feature of twitter - the messages.  Or more specifically, the message feeds.  A feed, also called an activity stream, is a time-based stream of messages that consists of all the messages posted by a user, as well as any messages posted by users they follow. Users want to see the content that matters to them and feeds are just that - personalized and relevant.

App Services makes it easy to use activity streams in your application. You need not worry about the complex back-end code as it is easily leveraged with simple API calls, enabling you to focus solely on the front end application.

Creating new activities

The foundation of activity streams is the activities collection. We use these activities as the containers for messages that are generated in our Messagee app. When a user posts a new message (or tweet), we simply add that message to an activity object and send it to the API.  To start, we need a simple form where the user will enter their message:

<form name="login" id="login-form">
<label for="content">Message (150 Chars max)</label>
<textarea id="content" class="input-xlarge" rows="3" style="margin: 0px; width: 100%; height: 125px; "></textarea>
</form>
<div style="width: 50%; float: left">
<button id="post-message">Post</button>
</div>

Like the previous forms, clicks on the submit button (post-message) are picked up by an event handler:

  $('#post-message').bind('click', postMessage);

The event handler calls the postMessage function which actually submits the activity to the API. First, the function gets a reference to the currently logged in user.  Then, using that user’s data, and the message that they are trying to post, an “actor” object is built and then saved to the API.  Upon success, the feed window is displayed:

  var options =
  {"actor" : {
    "displayName" : appUser.get('username'),
    "uuid" : appUser.get('uuid'),
    "username" : appUser.get('username'),
    "image" : {
      "duration" : 0,
      "height" : 80,
      "url" : "http://www.gravatar.com/avatar/",
      "width" : 80
    },
    "email" : appUser.get('email'),
    "picture": "fred"
  },
  "verb" : "post",
  "content" : $("#content").val(),
  "lat" : 48.856614,
  "lon" : 2.352222};

  client.createUserActivity('me', options, function(err, activity) { //first argument can be 'me', a uuid, or a username
    if (err) {
      alert('could not post message');
    } else {
       if (fullFeedView) {
        //reset the feed object so when we view it again, we will get the latest feed
        fullActivityFeed.resetPaging();
        showFullFeed();
      } else {
        //reset the feed object so when we view it again, we will get the latest feed
        userFeed.resetPaging();
        showMyFeed();
      }
      window.location = "#page-messages-list";
    }
  });

Note that we are not doing a POST operation directly to the activities collection.  We are posting to the activity to /users/me/activities.  This means that, along with creating the activity and adding it to the activities collection, we are  establishing a relationship between the user and the activity they are creating.  This is important because this “ownership” relationship is used in conjunction with “following” connections to determine which messages show up in a feed.

Making connections

The concept of following and followers, also known as a “social graph”, is what has made Twitter a social powerhouse.  Users follow other users so they can see the messages those other users post. For example, say we have two users, thing1 and thing2. Now suppose that thing1 is following thing2 (more about how to establish this relationship in just a bit). This means that thing1 cares about what thing2 has to say and wants to see all the messages that thing2 creates.  In other words, thing2’s messages should show up in thing1’s feed.

App Services has the facility for users to follow other users already built in.  We take advatage of it in Messagee by adding a “follow” link to each user’s posted message in the display. Clicking the link calls the followUser function. We use the /me shortcut as a reference to the currently logged in user to make the API call to create the following relationship:

  var options = {
    method:'POST',
    endpoint:'users/me/following/users/' + username
  };
  client.request(options, function (err, data) {
    if (err) {
      $('#now-following-text').html('Aw Shucks!  There was a problem trying to follow ' + username + '');
    } else {
      $('#now-following-text').html('Congratulations! You are now following ' + username + '');
      showMyFeed();
    }
  });

Upon a successful call to the API, a new “following” relationship between the current user and the user who posted the message. The API keeps track of all of these “following” relationships internally and automatically. Once the connection is made, the user’s “feed” can be queried to get the message stream. As we shall see, a user’s feed is made up of their messages as well as the messages of all the users they follow.

App Service Activity Streams can be used to model the message feeds found in Twitter.  Posting an Activity not only makes a new activity but also creates a connection between the activity and the user who posted it.  We also learned how to leverage the API to build a “social graph”, by creating a “following” relationship between one user and another. 


In this final segment, we see how to put the concepts together to create a customized feed that shows a user only the relevant data they care about, as well as how to get a list of all the activities in the application.

Full activities feed

After logging in, the main display in the Messagee app is an unfiltered activity stream that shows all the activities (messages) posted to the application. The view is shown by default, but can also be activated by clicking the “All Posts” button:

<a href="#" data-role="button" data-icon="grid" id="btn-show-full-feed" class="ui-btn-up-c">All Posts</a>

Clicking the button calls the showFullFeed function, which calls the get method of the fullActivityFeed Collection object:

  //make sure we are on the messages page
  window.location = "#page-messages-list";

  fullFeedView = true;
  $('#btn-show-full-feed').addClass('ui-btn-up-c');
  $('#btn-show-my-feed').removeClass('ui-btn-up-c');


  if  (fullActivityFeed) {
    fullActivityFeed.resetPaging();
    fullActivityFeed.fetch(function (err) {
      if (err) {
        alert('Could not get activity feed. Please try again.');
      } else {
        drawMessages(fullActivityFeed);
      }
    });
  } else {
    var options = {
      type:'activities',
      qs:{"ql":"order by created desc"}
    }
    //no feed obj yet, so make a new one
    client.createCollection(options, function(err, collectionObj){
      if (err) {
        alert('Could not get activity feed. Please try again.');
      } else {
        fullActivityFeed = collectionObj;
        drawMessages(fullActivityFeed);
      }
    });
  }

Upon success, it calls the drawMessages function which loops through all the returned results:

feed.resetEntityPointer();
   while(feed.hasNextEntity()) {
     var message = feed.getNextEntity();
     …

Once the messages are drawn to the screen, the success function also checks to see if there is a previous or next page of results:

fullActivityFeed.hasPreviousPage()

fullActivityFeed.hasNextPage()

If there are, then the appropriate buttons are displayed.

Customized feeds

The full view of all the Activities in the system is certainly useful, but what makes the app really interesting (and why Twitter is so successful) is that it also has the facility to show personalized feeds. When a user’s feed is queried, the API dynamically builds the feed based on any relevant relationships and returns the stream of messages.

To display the user’s feed, we have created a button:

<a href="#" data-role="button" data-icon="home" id="btn-show-my-feed">My Stream</a>

And attached it to a click event that calls the showMyFeed function:

  $('#btn-show-my-feed').bind('click', showMyFeed);

The showMyFeed function retrieves a paged list of all the messages in the user’s feed.  Notice that when we create the userfeed object, we again use the /me alias to make it easy to reference the currently logged in user:

  type:'user/me/feed'

Now, in the function, we can simply call fetch. The fetch method retrieves the list of messages from the server:  

  //make sure we are on the messages page
  window.location = "#page-messages-list";

  fullFeedView = false;
  $('#btn-show-full-feed').removeClass('ui-btn-up-c');
  $('#btn-show-my-feed').addClass('ui-btn-up-c');

  if  (userFeed) {
    userFeed.resetPaging();
    userFeed.fetch(function (err) {
      if (err) {
        alert('Could not get user feed. Please try again.');
      } else {
        drawMessages(userFeed);
      }
    });
  } else {
    //no feed obj yet, so make a new one
    var options = {
      type:'user/me/feed',
      qs:{"ql":"order by created desc"}
    }
    client.createCollection(options, function(err, collectionObj){
      if (err) {
       alert('Could not get user feed. Please try again.');
      } else {
        userFeed = collectionObj;
        drawMessages(userFeed);
      }
    });
  }

Just like the full feed, upon a successful call, the drawMessages function is called which loops through all the returned results:

feed.resetEntityPointer();
   while(feed.hasNextEntity()) {
     var message = feed.getNextEntity();
     …

Also, just like the full feed, once the messages are drawn to the screen, the success function checks to see if there is a previous or next page of results:

userFeed.hasPreviousPage()
userFeed.hasNextPage()

If there are more pages of data, then the appropriate buttons are displayed.


We've seen how to query and display the full activity stream as well as the more specialized feed for an individual user.  By combining these features with built-in user management and social graph functionality, a wide range of opportunities for socially relevant applications emerges.

We also saw how to use the JavaScript SDK to get up and running quickly.

Because the App Services API follows a RESTful design model, calls to invoke the API are simple to use, and because they take and return JSON, they work nicely with JavaScript. Put it all together and you have a platform that has enabled us to build a Twitter style app in a very short development cycle.