Exeami have search now!

Background

I always felt having a search in any website is useful, and I wanted to have one in mine. Even though I know how to exactly navigate to the content I want in my website, I sometimes feel lazy and wish I had a quick way to ask and get what I want.

The reasons why I want search are maybe obvious, but for some reason I had a feeling that it will be tough to implement and needs a lot of work till I actually did it. I had a fully functioning search which fits my requirements in less than a day of effort. That is the reason I wanted to write this blog to tell how easy it can be to have your own search in your website.

When I searched how to add to search to my website in a search engine, the most popular option is implementing search boxes from leading search engine providers. Most of the articles pointed out how to add these search boxes to my site. I leaned towards adding DuckDuckGo search box as I do not need to register to the service like google and as always I prefer DuckDuckGo. It took few minutes to add DuckDuckGo search box to my site, it works well, but I did not like the fact I will be redirected to DuckDuckGo. Pardon me with the metaphor, but it felt like I am leaving my home to search something from my home, and did it feel right. Then I realized that I need to implement custom search.

The next step in my search of adding search was to go and check in IndieWeb, where I found that Peter Molnar uses SQLite3 FTS4 engine for full text searching his site. That felt like a good way for me, but I did not get to see how he did it or any reference to his code. And I knew I had to do it my way, following the documentation.

The way I did it

On high level, my search is simple, I need a list of web pages to crawl, need to save the content of those pages in db, query the db and share the page link which matches my search query using FTS (Full Text Search).

Exeami was already structured with microformats as part of supporting Indie Web when I planned to implement search. So, I utilized this structure to crawl and store.

You can find the source code on Codeberg for micro search, named it "micro search" as its a search engine for sites supporting microformats2.

micro search exposes two api endpoints

• a PUT request that crawls the webiste, find h-entry and stores p-name, p-category, p-summery, e-content as a row item.
• a GET request that takes query parameter and returns the list of h-entry urls that matches the query text.

That's it, now Exeami have search functionality.