Collect Recipes From Your Favorite Websites

I started with the goal of creating a website that would scrape recipes from a few of my favorite food blogs. As a user, I wanted to quickly collect the ingredients from a recipe, so that I could save time on my way to the store.

I wanted to scrape the recipe title, image, and list of ingredients. To scrape a recipe, I needed to provide my scraper with the HTML elements that identified each of these items on the page. To support multiple websites, I built a `recipe_source` model that accepted these HTML identifiers as attributes.

As a user, all I needed to do was paste a recipe link into a form and hit submit. The scraper would first look at the domain and determine if that website was already supported (and it flashed a friendly error if not). Once the recipe_source was identified, the appropriate HTML attributes were inserted into the scraper, and it would run off to grab the recipe.

Once a recipe had been scraped, I needed a way to add multiple recipes to my grocery list. This was simple enough. I used Auth0 to allow users to sign in and then created a `recipe_user` join table. The grocery list view then looked at each recipe paired with the current user and listed out each ingredient from each recipe. I added a little javascript to allow users to check items off of their list.

The challenging part, however, was when an ingredient was shared between multiple recipes. I wanted to be able to group ingredients on the grocery list, but the scraper recorded the full ingredient text from the recipe — so “2 cups of butter” and “1 tbsp of butter” were interpreted as two different ingredients.

I decided to try to use Google’s Natural Language Processing to parse out the primary object of each ingredient string. About 80% of the time, it successfully plucked out “butter” from a string that says “2 cups of butter”.

Once I had the scraper working with 100+ food blogs, I started building out the website to create a way for people to discover recipes from other blogs.

Up until now, the scraper only plucked individual recipes when provided a link. I upgraded the scraper and built a rake task that would tell the scraper to go out to a blog and retrieve ALL recipes from that blog. Suddenly, I had tens of thousands of recipes in my database.

There were a few hundred people using the website at this point. When viewing a recipe, my website now displayed recipes that shared ingredients with that recipe. I also showed recipes that had been favorited by people who had favorited this recipe.

I started to use this website to organize my own recipes. I had saved several dozen recipes to Recidex, and the list was becoming difficult to scan through.

I built a collections feature that allowed me to group recipes by whatever label I liked. I created “Breakfast Recipes,” “Vegetarian Recipes,” etc. These recipe collections were also publicly visible, which offered yet another way for me to browse and discover recipes I might like.

To help drive adoption of my website, I wanted to create a button that blogs could paste on their website to encourage their readers to save their recipes.

It never quite worked, but I did manage to build a javascript snippet that could be inserted into a website and, when clicked, would open a new browser window to navigate to Recidex and save the recipe.