Building StreetCarts: Sample API Proxies to Access Food Truck Data
StreetCarts needs you!
We've got a few features we want to add, including a client app (maybe an Ionic or iOS app?), support for payment processing, and support for reviewing food carts (i.e., finishing the /reviews proxy).
If you're an Apigee developer and would like to participate, post a suggestion or question in the Apigee Community. Be sure to use the "streetcarts" tag when posting.
Back in October, Apigee's doc team started building an app using the technologies we write about every day. We recently finished the first version of StreetCarts, a set of sample proxies for accessing data about food carts (or food trucks, if you prefer).
You'll find the code in the GitHub repository we've put up, including instructions on how to get started, how to deploy the app to your Edge and API BaaS instances, and how to seed your data store and start experimenting.
Note: StreetCarts is a sample, not production-level code. Please don't try to run it as a production app!
In an October post on the Apigee Community, I said that doc folks often don't generally have time to cover more than the pat use cases. We usually doc a feature in isolation, rather than in combination with other features. But in the real world, combinations are what can make an app both interesting and tricky to code.
This post describes a few of the turns and reversals we experienced along the way while combining things to get the results we wanted.
StreetCarts includes the following technology features:
- Multiple client-facing proxies calling a single internal proxy (using proxy chaining) to share a connection with a backend resource
- Authentication combining Edge OAuth policies and an API BaaS data store
- Authorization combining OAuth scopes in Edge and permissions in API BaaS
- Node.js modules to integrate an API BaaS data store with Edge proxies
- A spike arrest policy to suppress potential traffic spikes
- Support for two distinct client access levels using the quota policy to enforce product-scoped quotas.
So what does it do?
We're putting together a list of the features for the next revision, but so far we’ve implemented these basic features:
- All users (even anonymous ones) can browse food carts, menus, and food items.
- Registered users who are owners can create/edit/delete food carts, menus, and food items.
- Registered users who are cart managers can create/edit/delete menus and food items.
We'd like to add the ability to purchase food items (in the example sense) down the road. Then again, we think it would be cool for others to code additional features and submit pull requests.
Could the code be more efficient? Without doubt. Are some of our design decisions debatable? Probably. But we're looking forward to turning the experience into more content. And we've had a good time learning how some of this stuff really works.
StreetCarts ended up with seven API proxies. Dividing the endpoints among these seven made sense from a functional standpoint. It also made it a little easier for us to work on them by helping us avoid collisions.
- Three proxies are for core data requests (such as creating or updating food carts, menus, or food items)
- One proxy is for user data operations
- One proxy is for OAuth token requests
- An "internal" proxy for accessing the data store is not accessible to clients. Its clients are the other StreetCarts proxies
- A proxy for reviews is not yet fully implemented
The /foodcarts proxy is the fattest, partly because it contains most of the POST endpoints. We wanted to be sure that we associated menus and items with their respective food carts. That was easier with a path such as the following, which is in the /foodcarts proxy:
If we'd had the menu creation endpoint in the /menus proxy, we'd have had to find another, maybe messier way to pass in the ID of the cart to which the new menu belonged.
GET, PUT, and DELETE requests for a single item or menu, on the other hand, are made to /items or /menus by specifying the entity ID.
Authentication and authorization comprise one of the areas where we had the most design churn. For example, early on we were using Edge product definitions to capture the difference between requests that guest (non-authenticated) users were authorized to make versus those allowed for registered users who were logged in.
In the end, as described below, we used OAuth scopes to define broad authorization categories and used products instead to define client access levels based on the number of requests allowed (quotas).
StreetCarts stores user data in an API BaaS data store and uses OAuth v2 for authentication. When a user authenticates, the accesstoken proxy calls the data-manager proxy to verify that the username and password correspond to a user account in the data store.
Because we're using the API BaaS token endpoint on the backend, we get an OAuth token back from BaaS. We store this token in the Edge-generated token we send back to the client. That way, every subsequent request can use the API BaaS-generated token to authenticate with the backend. That allowed us to use API BaaS permissions for small-grained authorization.
One of the challenges we had along the way was figuring out how much of the authorization logic should be in Edge versus in API BaaS. Was there a way to define user access only in a proxy and save a trip to the data store for invalid requests?
Closer to the front end of the request, in Edge, we defined OAuth scopes that established broad permissions. For example, a request from a user asking to update a food cart won't even reach the backend if they aren't in the owner.update scope. So it potentially cuts down a bit on the traffic to the backend by stopping requests in an area where the user doesn't belong.
At the start, we wanted to use API BaaS as a data store only. All of our logic around authentication and authorization, for example, would be in Edge. And the Node.js code we wrote to connect the front-end proxies to the backend data store would be swappable with other TBD connective code so that we could show how to use a different data store in another sample.
But it was a lot simpler to use API BaaS to get some non-DB things done, including parts of authentication and authorization. And trying to do all of that authorization in Edge felt like we were bending it to a shape it didn't like. In the end, we figured that if we were using a data store other than BaaS, we'd probably be using a separate user management system (like LDAP) for authorization. It just happened that BaaS includes both.
Another data store design decision was to use the API BaaS connections mechanism rather than query strings and primary/secondary keys. If you're not familiar with it, connections can look a bit like syntactic voodoo. But it helped us avoid writing queries against a Cassandra-based
API BaaS for capturing one-to-many associations (like food cart-to-menus and menu-to-food items).
One of the most challenging parts of this project was doing iterative development. Edge is a product that only runs on the server, with browser-based development tools also on the server. Yet we were keeping source code in a GitHub repository with local branches.
The development tools were handy, especially at the start. We could change code, then trace and make requests from a client to see what worked. But once something worked on a personal Edge server, the developer still needed to get the changed code into the repo. If they'd changed multiple files to get to that "working" place, they needed to figure out which files they'd changed, then somehow copy the changed code out of the management console into their local repo.
We each had a slightly different process, from comparing downloaded code against a local repo to deploying from source to test anew. Here's one process that seemed to work:
- Code and deploy your app to your personal Edge testing organization.
- Use a client (such as Postman) to make calls, tracing in Edge to discover where things fall down.
- Edit your local code to fix issues, then paste your local changes into the the Edge console's editing window. Save the changes in your personal Edge testing organization.
- Run and trace again.
- Repeat steps 2 through 4 as needed to get committable code.
- When you're happy with your changes, push your code to the repository (remember that you've been making changes to your local repo all along).
- Pull changes from others, then build and deploy to a shared staging environment for validation by the team.
StreetCarts needs your help!
We've got a short list of features we want to add. Some of the low-hanging fruit includes a client app (maybe an Ionic or iOS app?), support for payment processing, and support for reviewing food carts (i.e., finishing the /reviews proxy).
If you're a developer working with Apigee products, why not get involved? Let us know if you'd like to start working on a feature by posting a suggestion or question in the Apigee Community. Be sure to use the "streetcarts" tag when posting.
In the meantime, we'll post more in-depth content about the implementation in the Apigee Community.