How to insert and query documents into Azure Search

In this post, I’m going to share how you can insert data via the C# SDK library provided by Microsoft and also for those who are not using C# or with a .NET background, good old raw HTTP methods.

In my previous post Getting started with Azure Search, I talked about inserting data into a search index using Data Sources and Indexers. This is more of a pull model, as we are pulling data from a data source into an index. We can, however, use a push model where we are inserting data directly into an index which is what we are going to cover in this post.

The main benefit of pushing data is that it doesn’t matter what language you use and also where your data comes from.

Credit goes to my colleague Tim, who worked on this.

Full code can be found in my GitHub repo

Inserting data into a Search Index

Now, it’s worth mentioning there are few methods or shall we say modes in how we can push data. Azure refers to this as actions. These actions determine if we want to insert or update documents.

upload As it states, it is an action that will upload a document into the search index. It acts as an “upsert”. If the document doesn’t exist, it will insert it, otherwise, it will replace it if it does exist.
merge This will update an existing document. If the document does not exist, it will fail. You can also update only certain parts of a document and you do not need to supply the entire document to be updated.
mergeOrUpload This is a combination of the two above. It will either update a document or it will insert it.
delete This will remove the document from the index. You only need to specify the key of the document to delete.

More detailed information can be found on the Microsoft’s website about adding or deleting document actions.

Create a Search Index

Before we can insert or update data in an index, we will need to have a search index defined and created. I’m going to use the Search index that was created from my previous blog post.

Upload a document using C# .NET

Once we have a search index created and know the schema, we can go ahead and start inserting documents.

First, we need to add a reference to Microsoft.Azure.Search nuget package.

Using Visual Studio it will be:

Install-Package Microsoft.Azure.Search -Version 10.1.0

Via the command line:

dotnet add package Microsoft.Azure.Search --version 10.1.0

Then it’s a case of creating an instance of SearchServiceClient and specifying the search index name and credentials. For now, we are just going to upload documents.

Here is a code snippet below.

var serviceClient = new SearchServiceClient("demo-search-3jamlwx", new SearchCredentials("8585F9E40DBADDDCCB8B03CC74EB2B0C"));

var indexClient = serviceClient.Indexes.GetClient("star-wars-characters-index");
indexClient.SerializationSettings.ContractResolver = new CamelCasePropertyNamesContractResolver();

var actions = new IndexAction<Character>[]
{
    IndexAction.Upload( new Character
    {
        CharacterId = Guid.NewGuid(),
        FirstName = "Yoda",
        LastName = "",
        Gender = "male",
        EyeColor = "brown",
        Height = 66
    }),
    IndexAction.Upload( new Character
    {
        CharacterId = Guid.NewGuid(),
        FirstName = "Boba",
        LastName = "Frett",
        Gender = "male",
        EyeColor = "brown",
        Height = 183
    }),
    IndexAction.Upload(new Character
    {
        CharacterId = Guid.NewGuid(),
        FirstName = "Leia",
        LastName = "Organa",
        Gender = "female",
        EyeColor = "brown",
        Height = 150
    })
};

await indexClient.Documents.IndexAsync(IndexBatch.New(actions));

In the example above, I’m using the upload action, but it can be replaced with a merge or mergeOrUpload action depending on what type of action you want.

Alternative upload method

You may notice that in the above code snippet, we are creating an array of IndexAction. An alternative way and most commonly, you will already have a list of objects that you will want to upload. So, with the power of Linq, the above code snippet can be converted into something like this:

// create a standard list of objects
var characters = new List<Character>
{
    new Character
    {
        CharacterId = Guid.NewGuid(),
        FirstName = "Yoda",
        LastName = "",
        Gender = "male",
        EyeColor = "brown",
        Height = 66
    },
    new Character
    {
        CharacterId = Guid.NewGuid(),
        FirstName = "Boba",
        LastName = "Frett",
        Gender = "male",
        EyeColor = "brown",
        Height = 183
    },
    new Character
    {
        CharacterId = Guid.NewGuid(),
        FirstName = "Leia",
        LastName = "Organa",
        Gender = "female",
        EyeColor = "brown",
        Height = 150
    },
};

var mergeOrUploadActions = characters.Select(IndexAction.MergeOrUpload);

var indexBatch = IndexBatch.New(mergeOrUploadActions);

await searchIndexClient.Documents.IndexAsync(indexBatch);

Upload documents to an Index the Http way

So, in this example, we are going to do away with the C# and for use raw HTTP methods. To insert documents into an index, we can Post data into an index like so:

POST https://[servicename].search.windows.net/indexes/[index-name]/docs/index?api-version=[api-version]  
Content-Type: application/json
api-key: [admin key]

Here is an example of using the mergeOrUpload action of inserting data into the star-wars-characters-index search index.

POST https://demo-search-3jamlwx.search.windows.net/indexes/star-wars-characters-index/docs/index?api-version=2019-05-06
Content-Type: application/json  
api-key: 8585F9E40DBADDDCCB8B03CC74EB2B0C

Example JSON body:

{
    "value": [
        {
            "@search.action" : "mergeOrUpload",
            "characterId": "dc5c2955-653e-45c3-a70a-8022302aec83",
            "firstName": "Yoda",
            "lastName": "",
            "gender": "male",
            "eyeColor": "brown",
            "height": 66
        },
        {
            "@search.action" : "mergeOrUpload",
            "characterId": "8569c8c8-ea31-4504-b7b1-a4240a6d4881",
            "firstName": "Boba",
            "lastName": "Frett",
            "gender": "male",
            "eyeColor": "brown",
            "height": 183
        },
        {
            "@search.action" : "mergeOrUpload",
            "characterId": "57d71d79-e6f3-4c25-937a-f71ce59c2f18",
            "firstName": "Leia",
            "lastName": "Organa",
            "gender": "female",
            "eyeColor": "brown",
            "height": 150
        }
    ]
}

Query Azure Search

Now that we have some documents in Azure search index, the next step is to fetch them.

Again, we’ll go through how to use the C# SDK library, followed by just raw HTTP methods.

Query Azure Search using C# .NET

In the example of uploading a document using C#, it shows you how to obtain an instance of SearchServiceClient. Once you have this, you can simply call the Search method and pass in a search query. To get all documents, you can just use the asterix * character.

 var searchResults = await searchIndexClient.Documents.SearchAsync<Character>("*");

To write a little more advanced query, say you want to filter by gender, you can use a filter expression such as gender eq 'female'. This is using OData notation, which more information can be found at Microsoft’s Query OData language overview..

 var searchResults = await searchIndexClient.Documents.SearchAsync<Character>("gender eq 'female'");

Query Azure Search using HTTP

The same can be achieved using Azures REST API.

To query, we can use a GET using the URL below and specifying the search query in the search query string.

GET https://[servicename].search.windows.net/indexes/[index-name]/docs?api-version=[api-version]&search[query]
Content-Type: application/json
api-key: [admin key]

Here’s an example:

GET https://demo-search-3jamlwx.search.windows.net/indexes/star-wars-characters-index/docs?api-version=2019-05-06&search=*

Example response:

{
    "@odata.context": "https://demo-search-3jamlwx.search.windows.net/indexes('star-wars-characters-index')/$metadata#docs(*)",
    "value": [
        {
            "@search.score": 1.0,
            "characterId": "adb81166-87d7-2eb3-d103-4ca6a84327c4",
            "firstName": "Wilhuff",
            "lastName": "Tarkin",
            "eyeColor": "blue",
            "gender": "male",
            "height": 180
        },
        {
            "@search.score": 1.0,
            "characterId": "b616614c-1adb-0bac-59e8-59af27a7e631",
            "firstName": "Beru",
            "lastName": "Whitesun Lars",
            "eyeColor": "blue",
            "gender": "female",
            "height": 165
        },
        ......
}

Alternative POST method

You can also use the POST method as sometimes providing complex search queries and filters in the URL can be very difficult to read. Instead, it can be sent as the body of the request like so.

POST https://[servicename].search.windows.net/indexes/[index-name]/docs/search?api-version=[api-version]
Content-Type: application/json
api-key: [admin key]
Json Body:
{
    "search": "[search query]",
    "filter": "[OData filter expression]",
    "count": "true/false"
}

Here is another example:

POST https://demo-search-3jamlwx.search.windows.net/indexes/star-wars-characters-index/docs/search?api-version=2019-05-06
Content-Type: application/json  
api-key: 8585F9E40DBADDDCCB8B03CC74EB2B0C

{
      "search": "*",
      "filter": "gender eq 'female'",
      "count": "true"
}

Summary

I hope this article gives you enough to get started with Azure Search. It only covers some of the basics of inserting, querying, but at the very least it should get you going and give you a foundation to start with.

The C# code samples above can be found in my GitHub Repo.

Inserting data into a Search Index#

Create a Search Index#

Upload a document using C# .NET#

Alternative upload method#

Upload documents to an Index the Http way#

Query Azure Search#

Query Azure Search using C# .NET#

Query Azure Search using HTTP#

Alternative POST method#

Summary#