Suppose we create a search engine from the very beginning. So what do we have to do?
In the first place, we have to create a system of crawling millions of websites regularly. Crawl means reading a website’s content with a program. Then you will have to arrange the contents of that website again. The job can not be done well with the simple database, but there is another way to think.
Now, let’s say someone searches for “laptop”. We will show any website number 1, which website will be shown in 2 numbers? So for the keywords, we have to make a ranking algorithm and implement it. And when searching, we need to indexing very fast results. Because if you do not search then the results will take a few hours.
Then we have to do some work with natural language processing. So that people can easily write search queries.
When Google’s founder Sergey Brin and Larry Page created Google search engine in 1998, they published a paper with that. Click here to read it.
Apache Lussin is currently among the most popular open source search engines. If you go to this website, you can know about Lusin.
And the only search engine in the Bangla language Pipilika.
To find out more about search engines, search engines will have to search.