Maximizing the Potential of Your Laravel Blog Project with a Sitemap

What is a Sitemap?

A sitemap is a file that lists the pages of a website and provides additional information about each page, such as when it was last updated and how important it is in relation to other pages on the site. Sitemaps are used to help search engines discover and crawl the pages on a website, and they can also be used to give website visitors an overview of the content available on the site.

There are two main types of sitemaps:

• HTML sitemaps, which are designed to be read by humans and provide a hierarchical view of the website's content.
• XML sitemaps, which are designed to be read by search engines and provide more detailed information about each page, such as the URL, the last modification date, and the page's relative importance within the website. For an example, take a look at mine here.

Sitemaps can be useful for websites of all sizes, but they are especially important for large websites with a lot of pages or for websites with content that is frequently updated - such as a blog. By providing search engines with a list of all the pages on a website, sitemaps can help ensure that all of a website's content is properly indexed and ranked by search engines.

Creating a Dynamic XML Sitemap in Laravel

In this example, we will create a dynamic XML sitemap in Laravel that is automatically updated as additional blog posts are added. We will assume that the Laravel project has already been set up. We will cover all the necessary steps, including creating a controller, setting up a route to handle the sitemap, and creating a blade view for the sitemap. Lastly, we will take a look at an optional addition that can make indexing your website faster and easier on Google.

Step 1: Creating the Sitemap Controller

By creating a sitemap controller, we can centralize the logic for generating and serving the sitemap in a single location, making it easier to manage and maintain. Let's go ahead and create the controller using the following artisan command.

php artisan make:controller SitemapXmlController

With our new controller ready, go ahead and add the following logic to the index method. We will be returning a view called "sitemap" along with all of our posts. I do this through my Post model, and I sort by latest. You will need to use your own model, of course. Make sure you include the necessary headers.

namespace App\Http\Controllers;
    
use App\Models\Post;
use Illuminate\Http\Request;

class SitemapXmlController extends Controller
{
    public function index()
    {
        return response()->view('sitemap', [
            'posts' => Post::latest('created_at')->get(),
        ])->header('Content-Type', 'text/xml');
    }
}

Step 2: Registering The Route

The next step is to set up a route to handle the sitemap. This route will be responsible for generating the sitemap and serving it to search engines when they request it. In your web.php file, add the following line and make sure to import your controller.

use App\Http\Controllers\SitemapXmlController;

Route::get('sitemap.xml', [SitemapXmlController::class, 'index']);

Step 3: Sitemap Blade File

Now that we have our sitemap controller set up, the next step is to create the blade view that will provide the sitemap data to users and search engines. Create a blade file under the following file path and populate the file with the provided code.

<?php echo '<?xml version="1.0" encoding="UTF-8"?>'; ?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    @aforeach ($posts as $post)
        <url>
            <loc>{{ url('/') }}/posts/{{ $post->slug }}</loc>
            <lastmod>{{ $post->created_at->tz('UTC')->toAtomString() }}</lastmod>
            <changefreq>weekly</changefreq>
            <priority>0.8</priority>
        </url>
    @aendforeach
</urlset>

In an XML sitemap, the "priority" attribute is a value between 0.0 and 1.0 that indicates the relative importance of a particular page on the website. This value is used by search engines to determine how they should prioritize crawling and indexing the page. In general, it's a good idea to set the priority value for the most important pages on your website to a higher value (closer to 1.0) and the priority value for less important pages to a lower value (closer to 0.0). However, you should not set the priority value for all of your pages to 1.0, as this can indicate to search engines that all of the pages on your website are equally important, which is not likely to be the case.

Note that in this code, my posts are accessed through the url farzanyaz.com/posts/post-slug. Yours may be different, so make sure it's a valid link structure.

Step 4: Start Laravel Application

We're done! Now simply run your Laravel application with the following command and then head over to your browser to view the local sitemap. If you have published posts, they should appear on your sitemap. To see what this should look like, check out my sitemap here.

php artisan serve

http://127.0.0.1:8000/sitemap.xml

Step 5: Update Robots.txt

A robots.txt file is a text file that is placed in the root directory of a website and is used to communicate with web crawlers (also known as "bots" or "spiders") that visit the website. The purpose of the robots.txt file is to instruct web crawlers what pages or files on the website they should or should not crawl. You may want to add your sitemap to this file, and ensure that it is not restricting the pages you want displayed. Here is what my robots.txt file looks like.

Sitemap: https://farzanyaz.com/sitemap.xml
User-agent: *
Disallow:

How to Ping Google about Sitemap Updates

By informing search engines about updates to your sitemap, you can help ensure that they are aware of all the pages on your website and that they are able to properly index and rank them. This is especially important for websites with a lot of pages or for websites with content that is frequently updated - such as a blog with new posts! Keep in mind that you will need to resubmit your sitemap every time you make changes to it.

Option 1: Manual Submittion

To manually submit your updated sitemap to Google, you will need to have a Google Search Console account and verify ownership of your website. Once you have set up your account and verified ownership of your website, you can follow these steps to submit your sitemap:

• Go to the Google Search Console dashboard for your website.
• In the left-hand menu, click on "Sitemaps" under the "Index" section.
• In the "Add a new sitemap" field, enter the URL of your sitemap and submit.

It may take some time for Google to process and index the pages in your sitemap. You can check the status of your sitemap by going to the "Sitemaps" page in Google Search Console and looking at the "Status" column. If you see any errors or issues with your sitemap, you can troubleshoot them by clicking on the "Details" link next to the sitemap.

Option 2: Semi-Automatic Submittion

With a short script, we can automatically ping Google that our sitemap has been updated. Add the following code somewhere appropriate and call it when you need - perhaps after a post is created, or make it a seperate route that you can call whenever you wish. Pinging once is enough, and any subsequent pings will have no effect on how fast your new posts are crawled.

public function ping()
{
    $sitemapUrl = "Your sitemap url" // Example: "http://www.farzanyaz.com/sitemap.xml";

    function myCurl($url){
        $ch = curl_init($url);
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_exec($ch);
        $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
        curl_close($ch);
        return $httpCode;
    }

    // Pinging Google
    $url = "http://www.google.com/webmasters/sitemaps/ping?sitemap=".$sitemapUrl;
    $returnCode = myCurl($url);
    dump( "
Google Sitemaps has been pinged (return code: $returnCode).
");
}

Conclusion

There we have it! We covered what a sitemap is, what they are used for, and how to set up a dynamic sitemap for a Laravel blog project. Lastly, we covered how you can inform Google that your sitemap has been updated. A sitemap that features all of your posts will ensure that search engines index and crawl all of your content that you worked so hard to create!