Building a Recipe Search Site with Angular and Elasticsearch
2016-12-25 08:14
531 查看
https://www.sitepoint.com/building-recipe-search-site-angular-elasticsearch/
By Adam
Bard April 15, 2014
Have you ever wanted to build a search feature into an application? In the old days, you might have found yourself wrangling with Solr,
or building your own search service on top of Lucene —
if you were lucky. But, since 2010, there’s been an easier way: Elasticsearch.
Elasticsearch is an open-source storage engine built on Lucene. It’s more than a search engine; it’s a true document store, albeit one emphasizing search performance over consistency or durability. This means that, for many applications, you can use Elasticsearch
as your entire backend. Applications such as…
In this article, you’ll learn how to use Elasticsearch with AngularJS to
create a search engine for recipes, just like the one atOpenRecipeSearch.com.
Why recipes?
OpenRecipes exists,
which makes our job a lot easier.
Why not?
OpenRecipes is an open-source project that scrapes a bunch of recipe sites for recipes, then provides them for download in a handy JSON format. That’s great for us, because Elasticsearch uses JSON too. However, we have to get Elasticsearch up and running before
we can feed it all those recipes.
Download Elasticsearch and
unzip it into whatever directory you like. Next, open a terminal,
the directory you just unzipped, and run
Windows). Ta-da! You’ve just started your very own elasticsearch instance. Leave that running while you follow along.
One of the great features of Elasticsearch is its out-of-the-box RESTful backend, which makes it easy to interact with from many environments. We’ll be using the JavaScript
driver, but you could use whichever
one you like; the code is going to look very similar either way. If you like, you can refer to this handy
reference (disclaimer: written by me).
Now, you’ll need a copy of the OpenRecipes
database. It’s just a big file full of JSON documents, so it’s straightfoward to write a quick Node.js script to get them in there. You’ll need to get the JavaScript Elasticsearch library for this, so run
and add the following code.
Next, run the script using the command
you have it handy:
Now, you might be OK using
search for recipes, but if the world is going to love your recipe search, you’ll need to…
This is where Angular comes in. I chose Angular for two reasons: because I wanted to, and because Elasticsearch’s JavaScript library comes with an experimental Angular adapter. I’ll leave the design as an exercise to the reader, but I’ll show you the important
parts of the HTML.
Get your hands on Angular and Elasticsearch now. I recommend Bower,
but you can just download them too. Open your
and insert them wherever you usually put your JavaScript (I prefer just before the closing
myself, but that’s a whole other argument):
Now, let’s stop to think about how our app is going to work:
The user enters a query.
We send the query as a search to Elasticsearch.
We retrieve the results.
We render the results for the user.
The following code sample shows the key HTML for our search engine, with Angular directives in place. If you’ve never used Angular, that’s OK. You only need to know a few things to grok this example:
HTML attributes starting with
Angular directives.
The dynamic parts of your Angular applications are enclosed with an
an
need to be on the same element, but they can be.
All other references to variables in the HTML refer to properties on the
that we’ll meet in the JavaScript.
The parts enclosed in
template variables, like in Django/Jinja2/Liquid/Mustache templates.
Now, we can start writing our JavaScript. We’ll start with the module, which we decided above would be called
the
For those new to Angular, the
our handler function so we can use it. This system of dependency injection removes the need for us to rely on global variables (except the global
the
just created).
Next, we’ll write the controller, named
We need to make sure to initialize the
and
used in the template, as well as providing
actions.
You should recognize everything on the
from the HTML. Notice that our actual search query relies on a mysterious object called
A service is Angular’s way of providing reusable utilities for doing things like talking to outside resources. Unfortunately, Angular doesn’t provide
so we’ll have to write it ourselves. Here’s what it looks like:
Our service is quite barebones. It exposes a single method,
that allows us to send a query to Elasticsearch’s, searching across all fields for the given term. You can see that in the
in the body of the call to
a special keyword that lets us search all fields. If instead, our query was
The results come back in order of decreasing “score”, which is Elasticsearch’s guess at the document’s relevance based on keyword frequency and placement. For a more complicated search, we could tune the relative weights of the score (i.e. a hit in the title
is worth more than in the description), but the default seems to do pretty well without it.
You’ll also notice that the search accepts an
Since the results are ordered, we can use this to fetch more results if requested by telling Elasticsearch to skip the first n results.
Deployment is a bit beyond the scope of this article, but if you want to take your recipe search live, you need to be careful. Elasticsearch has no concept of users or permissions. If you want to prevent just anyone from adding or deleting recipes, you’ll need
to find some way to prevent access to those REST endpoints on your Elasticsearch instance. For example,OpenRecipeSearch.com uses
nginx as a proxy in front of Elasticsearch to prevent outside access to all endpoints but
Now, if you open
a browser, you should see an unstyled list of recipes, since our controller fetches some randomly for you on page load. If you enter a new search, you’ll get 10 results relating to whatever you searched for, and if you click “More…” at the bottom of the page,
some more recipes should appear (if there are indeed more recipes to fetch).
That’s all there is to it! You can find all the necessary files to run this project on GitHub.
By Adam
Bard April 15, 2014
Have you ever wanted to build a search feature into an application? In the old days, you might have found yourself wrangling with Solr,
or building your own search service on top of Lucene —
if you were lucky. But, since 2010, there’s been an easier way: Elasticsearch.
Elasticsearch is an open-source storage engine built on Lucene. It’s more than a search engine; it’s a true document store, albeit one emphasizing search performance over consistency or durability. This means that, for many applications, you can use Elasticsearch
as your entire backend. Applications such as…
Building a Recipe Search Engine
In this article, you’ll learn how to use Elasticsearch with AngularJS tocreate a search engine for recipes, just like the one atOpenRecipeSearch.com.
Why recipes?
OpenRecipes exists,
which makes our job a lot easier.
Why not?
OpenRecipes is an open-source project that scrapes a bunch of recipe sites for recipes, then provides them for download in a handy JSON format. That’s great for us, because Elasticsearch uses JSON too. However, we have to get Elasticsearch up and running before
we can feed it all those recipes.
Download Elasticsearch and
unzip it into whatever directory you like. Next, open a terminal,
cdto
the directory you just unzipped, and run
bin/elasticsearch(
bin/elasticsearch.baton
Windows). Ta-da! You’ve just started your very own elasticsearch instance. Leave that running while you follow along.
One of the great features of Elasticsearch is its out-of-the-box RESTful backend, which makes it easy to interact with from many environments. We’ll be using the JavaScript
driver, but you could use whichever
one you like; the code is going to look very similar either way. If you like, you can refer to this handy
reference (disclaimer: written by me).
Now, you’ll need a copy of the OpenRecipes
database. It’s just a big file full of JSON documents, so it’s straightfoward to write a quick Node.js script to get them in there. You’ll need to get the JavaScript Elasticsearch library for this, so run
npm install elasticsearch. Then, create a file named
load_recipes.js,
and add the following code.
var fs = require('fs'); var es = require('elasticsearch'); var client = new es.Client({ host: 'localhost:9200' }); fs.readFile('recipeitems-latest.json', {encoding: 'utf-8'}, function(err, data) { if (err) { throw err; } // Build up a giant bulk request for elasticsearch. bulk_request = data.split('\n').reduce(function(bulk_request, line) { var obj, recipe; try { obj = JSON.parse(line); } catch(e) { console.log('Done reading'); return bulk_request; } // Rework the data slightly recipe = { id: obj._id.$oid, // Was originally a mongodb entry name: obj.name, source: obj.source, url: obj.url, recipeYield: obj.recipeYield, ingredients: obj.ingredients.split('\n'), prepTime: obj.prepTime, cookTime: obj.cookTime, datePublished: obj.datePublished, description: obj.description }; bulk_request.push({index: {_index: 'recipes', _type: 'recipe', _id: recipe.id}}); bulk_request.push(recipe); return bulk_request; }, []); // A little voodoo to simulate synchronous insert var busy = false; var callback = function(err, resp) { if (err) { console.log(err); } busy = false; }; // Recursively whittle away at bulk_request, 1000 at a time. var perhaps_insert = function(){ if (!busy) { busy = true; client.bulk({ body: bulk_request.slice(0, 1000) }, callback); bulk_request = bulk_request.slice(1000); console.log(bulk_request.length); } if (bulk_request.length > 0) { setTimeout(perhaps_insert, 10); } else { console.log('Inserted all records.'); } }; perhaps_insert(); });
Next, run the script using the command
node load_recipes.js. 160,000 records later, we have a full database of recipes ready to go. Check it out with
curlif
you have it handy:
$ curl -XPOST http://localhost:9200/recipes/recipe/_search -d '{"query": {"match": {"_all": "cake"}}}'
Now, you might be OK using
curlto
search for recipes, but if the world is going to love your recipe search, you’ll need to…
Build a Recipe Search UI
This is where Angular comes in. I chose Angular for two reasons: because I wanted to, and because Elasticsearch’s JavaScript library comes with an experimental Angular adapter. I’ll leave the design as an exercise to the reader, but I’ll show you the importantparts of the HTML.
Get your hands on Angular and Elasticsearch now. I recommend Bower,
but you can just download them too. Open your
index.htmlfile
and insert them wherever you usually put your JavaScript (I prefer just before the closing
bodytag
myself, but that’s a whole other argument):
<script src="path/to/angular/angular.js"></script> <script src="path/to/elasticsearch/elasticsearch.angular.js"></script>
Now, let’s stop to think about how our app is going to work:
The user enters a query.
We send the query as a search to Elasticsearch.
We retrieve the results.
We render the results for the user.
The following code sample shows the key HTML for our search engine, with Angular directives in place. If you’ve never used Angular, that’s OK. You only need to know a few things to grok this example:
HTML attributes starting with
ngare
Angular directives.
The dynamic parts of your Angular applications are enclosed with an
ng-appand
an
ng-controller.
ng-appand
ng-controllerdon’t
need to be on the same element, but they can be.
All other references to variables in the HTML refer to properties on the
$scopeobject
that we’ll meet in the JavaScript.
The parts enclosed in
{{}}are
template variables, like in Django/Jinja2/Liquid/Mustache templates.
<div ng-app="myOpenRecipes" ng-controller="recipeCtrl"> <!-- The search box puts the term into $scope.searchTerm and calls $scope.search() on submit --> <section class="searchField"> <form ng-submit="search()"> <input type="text" ng-model="searchTerm"> <input type="submit" value="Search for recipes"> </form> </section> <!-- In results, we show a message if there are no results, and a list of results otherwise. --> <section class="results"> <div class="no-recipes" ng-hide="recipes.length">No results</div> <!-- We show one of these elements for each recipe in $scope.recipes. The ng-cloak directive prevents our templates from showing on load. --> <article class="recipe" ng-repeat="recipe in recipes" ng-cloak> <h2> <a ng-href="{{recipe.url}}">{{recipe.name}}</a> </h2> <ul> <li ng-repeat="ingredient in recipe.ingredients">{{ ingredient }}</li> </ul> <p> {{recipe.description}} <a ng-href="{{recipe.url}}">... more at {{recipe.source}}</a> </p> </article> <!-- We put a link that calls $scope.loadMore to load more recipes and append them to the results.--> <div class="load-more" ng-hide="allResults" ng-cloak> <a ng-click="loadMore()">More...</a> </div> </section>
Now, we can start writing our JavaScript. We’ll start with the module, which we decided above would be called
myOpenRecipes(via
the
ng-appattribute).
/** * Create the module. Set it up to use html5 mode. */ window.MyOpenRecipes = angular.module('myOpenRecipes', ['elasticsearch'], ['$locationProvider', function($locationProvider) { $locationProvider.html5Mode(true); }] );
For those new to Angular, the
['$locationProvider', function($locationProvider) {...}]business is our way of telling Angular that we’d like it to pass
$locationProviderto
our handler function so we can use it. This system of dependency injection removes the need for us to rely on global variables (except the global
angularand
the
MyOpenRecipeswe
just created).
Next, we’ll write the controller, named
recipeCtrl.
We need to make sure to initialize the
recipes,
allResults,
and
searchTermvariables
used in the template, as well as providing
search()and
loadMore()as
actions.
/** * Create a controller to interact with the UI. */ MyOpenRecipes.controller('recipeCtrl', ['recipeService', '$scope', '$location', function(recipes, $scope, $location) { // Provide some nice initial choices var initChoices = [ "rendang", "nasi goreng", "pad thai", "pizza", "lasagne", "ice cream", "schnitzel", "hummous" ]; var idx = Math.floor(Math.random() * initChoices.length); // Initialize the scope defaults. $scope.recipes = []; // An array of recipe results to display $scope.page = 0; // A counter to keep track of our current page $scope.allResults = false; // Whether or not all results have been found. // And, a random search term to start if none was present on page load. $scope.searchTerm = $location.search().q || initChoices[idx]; /** * A fresh search. Reset the scope variables to their defaults, set * the q query parameter, and load more results. */ $scope.search = function() { $scope.page = 0; $scope.recipes = []; $scope.allResults = false; $location.search({'q': $scope.searchTerm}); $scope.loadMore(); }; /** * Load the next page of results, incrementing the page counter. * When query is finished, push results onto $scope.recipes and decide * whether all results have been returned (i.e. were 10 results returned?) */ $scope.loadMore = function() { recipes.search($scope.searchTerm, $scope.page++).then(function(results) { if (results.length !== 10) { $scope.allResults = true; } var ii = 0; for (; ii < results.length; ii++) { $scope.recipes.push(results[ii]); } }); }; // Load results on first run $scope.loadMore(); }]);
You should recognize everything on the
$scopeobject
from the HTML. Notice that our actual search query relies on a mysterious object called
recipeService.
A service is Angular’s way of providing reusable utilities for doing things like talking to outside resources. Unfortunately, Angular doesn’t provide
recipeService,
so we’ll have to write it ourselves. Here’s what it looks like:
MyOpenRecipes.factory('recipeService', ['$q', 'esFactory', '$location', function($q, elasticsearch, $location) { var client = elasticsearch({ host: $location.host() + ':9200' }); /** * Given a term and an offset, load another round of 10 recipes. * * Returns a promise. */ var search = function(term, offset) { var deferred = $q.defer(); var query = { match: { _all: term } }; client.search({ index: 'recipes', type: 'recipe', body: { size: 10, from: (offset || 0) * 10, query: query } }).then(function(result) { var ii = 0, hits_in, hits_out = []; hits_in = (result.hits || {}).hits || []; for(; ii < hits_in.length; ii++) { hits_out.push(hits_in[ii]._source); } deferred.resolve(hits_out); }, deferred.reject); return deferred.promise; }; // Since this is a factory method, we return an object representing the actual service. return { search: search }; }]);
Our service is quite barebones. It exposes a single method,
search(),
that allows us to send a query to Elasticsearch’s, searching across all fields for the given term. You can see that in the
querypassed
in the body of the call to
search:
{"match": {"_all": term}}.
_allis
a special keyword that lets us search all fields. If instead, our query was
{"match": {"title": term}}, we would only see recipes that contained the search term in the title.
The results come back in order of decreasing “score”, which is Elasticsearch’s guess at the document’s relevance based on keyword frequency and placement. For a more complicated search, we could tune the relative weights of the score (i.e. a hit in the title
is worth more than in the description), but the default seems to do pretty well without it.
You’ll also notice that the search accepts an
offsetargument.
Since the results are ordered, we can use this to fetch more results if requested by telling Elasticsearch to skip the first n results.
Some Notes on Deployment
Deployment is a bit beyond the scope of this article, but if you want to take your recipe search live, you need to be careful. Elasticsearch has no concept of users or permissions. If you want to prevent just anyone from adding or deleting recipes, you’ll needto find some way to prevent access to those REST endpoints on your Elasticsearch instance. For example,OpenRecipeSearch.com uses
nginx as a proxy in front of Elasticsearch to prevent outside access to all endpoints but
recipes/recipe/_search.
Congratulations, You’ve Made a Recipe Search
Now, if you open index.htmlin
a browser, you should see an unstyled list of recipes, since our controller fetches some randomly for you on page load. If you enter a new search, you’ll get 10 results relating to whatever you searched for, and if you click “More…” at the bottom of the page,
some more recipes should appear (if there are indeed more recipes to fetch).
That’s all there is to it! You can find all the necessary files to run this project on GitHub.
相关文章推荐
- Search Engine Optimization -Building Traffic and Making Money with SEO
- Recipe: Dynamic Site Layout and Style Personalization with ASP.NET (转)
- Manage Spring Boot Logs with Elasticsearch, Logstash and Kibana
- Visualizing data with Elasticsearch, Logstash and Kibana
- 用Python和OpenCV创建一个图片搜索引擎的完整指南 The complete guide to building an image search engine with Python and
- Get Started with ElasticSearch and Wicket
- elasticsearch-head is a web front end for browsing and interacting with an Elastic Search cluster.
- How to Build a Search Page with Elasticsearch and .NET
- a simple search with AngularJS and PHP
- Dynamically Loading Controllers and Views with AngularJS/$controllerProvider and RequireJS
- ASP.NET Web API 2 external logins with Facebook and Google in AngularJS app
- Kivy: Building GUI and Mobile apps with Python
- 初步了解Angular 2端到端的测试 Introduction to E2E Testing with the Angular CLI and Protractor
- GSL 1.15 and 1.16 building with Visual Studio 2010 --FROM 4fire
- Building and Installing ACE on Win32 with MinGW/ MSYS
- elasticsearch query and
- Getting Started with AngularJS 1.5 and ES6: part 6
- Mastering the game of Go with deep neural networks and tree search
- Building Faster APIs with NodeJs and Redis
- Introducing WF4.0: Building Distributed Apps with WF 4.0 and WF 4.0 Services