Sitemap Libraries - Page 6

:page_with_curl: Robots.txt reader, parser and matcher.

JavaScript 7Updated: 5 y ago License: Permissive (MIT)

An Ember addon that automatically generates a project sitemap.xml file

JavaScript 7Updated: 3 y ago License: Permissive (MIT)

A plugin for Vuepress to create the robots.txt file 用来生成robots.txt文件的Vuepress插件

JavaScript 7Updated: 4 y ago License: No License (No License)

A Sitemap plugin for Croogo Cms

PHP 7Updated: 7 y ago License: No License (No License)

XML sitemap generator for Textpattern CMS

PHP 7Updated: 3 y ago License: Strong Copyleft (GPL-2.0)

A sitemap implementation for AnchorCMS 0.9.2

PHP 7Updated: 5 y ago License: No License (No License)

Generate a basic sitemap.xml for your Known site

PHP 7Updated: 5 y ago License: No License (No License)

Dynamically creates sitemap.xml for ASP.Net Core Razor pages.

C# 7Updated: 4 y ago License: Permissive (MIT)

Crawl given URL and generate sitemap.xml file

PHP 7Updated: 4 y ago License: Permissive (MIT)

A friendly tool for creating dynamic robots.txt files in Umbraco

C# 7Updated: 3 y ago License: Permissive (MIT)

使用cefsharp抓取网站后台数据。用xml和javascript对爬虫进行配置。你需要自己写一个socket或webapi对抓取到的html进行分析和存储，下载后请认真阅读Readme.txt

C# 7Updated: 4 y ago License: No License (No License)

Creator Sitemap with PHP

PHP 7Updated: 6 y ago License: No License (No License)

Python3 package for writing large XML sitemaps with no external dependencies

Python 7Updated: 2 y ago License: Permissive (MIT)

Helps to create routing robots.txt and sitemap.xml for asp.net core project.

C# 7Updated: 4 y ago License: Permissive (MIT)

Monitor and report changes across one or more robots.txt files.

Python 7Updated: 4 y ago License: Permissive (MIT)

You can visit https://www.steadylearner.com/sitemap.xml or sitemap.txt

Rust 7Updated: 5 y ago License: No License (No License)

A tool to generate sitemap. 支持baidu等国内sitemap格式。支持自动ping搜索引擎，即自动提交sitemap文件到搜索引擎。

Go 7Updated: 5 y ago License: No License (No License)

A web-based system to manage office hours

Scala 7Updated: 4 y ago License: Strong Copyleft (GPL-3.0)

A dead simple site-map plugin for the awesome Dancer framework.

Perl 7Updated: 10 y ago License: No License (No License)

This repository hosts and generates site maps for GitHub Wiki SEE.

Python 7Updated: 3 y ago License: Permissive (MIT)

typecho-sitemap.xml全站百度收录索引地图插件

PHP 7Updated: 3 y ago License: No License (No License)

资料收集于互联网

JavaScript 7Updated: 2 y ago License: No License (No License)

Support for Robots Exclusion Protocol, including parsing and matching against robot.txt directives.

Java 6Updated: 4 y ago License: Permissive (BSD-3-Clause)

Auto-Generate your Sitemap.xml with Python and deploy a Cron job to Google Cloud.

Python 6Updated: 4 y ago License: No License (No License)

一款可以用来检测舆情的爬虫，自定义框架编写 支持搜索模式的不通逻辑组合 支持指定深度爬取 支持js爬取 支持数据库存取 支持导出excel 支持中英文 自动分类 详细操作看使用说明.txt

Python 6Updated: 4 y ago License: No License (No License)

Sitemap plugin for Mephisto Edge (Engine based plugin)

Ruby 6Updated: 5 y ago License: Proprietary (Proprietary)

AngularJS && SPA Application Sitemap Generator

JavaScript 6Updated: 5 y ago License: No License (No License)

Generate a robots.txt for a web project (uses the robots.txt from HTML5 Boilerplate). This can be run from the command line when Generate is installed globally, or used as a plugin in another generator.

JavaScript 6Updated: 7 y ago License: Permissive (MIT)

nodejs爬虫，输入网站自动生成网站sitemap

JavaScript 6Updated: 4 y ago License: No License (No License)

Allows you to generate a sitemap page.

PHP 6Updated: 5 y ago License: Permissive (BSD-3-Clause)

The Sitemap sample project showcases how to create a sitemap module that generates a search engine-friendly sitemap.xml file for your site. As a result, you can enjoy the SEO benefits of letting search engines know about your site structure.

C# 6Updated: 5 y ago License: No License (No License)

Submit unlimited sitemaps to Google Webmaster Tools at once - with just one single call: new GWT_SitemapsUpload($sitemaps);

PHP 6Updated: 4 y ago License: No License (No License)

jQuery + PHP, free sitemap.xml generator

PHP 6Updated: 4 y ago License: Strong Copyleft (GPL-2.0)

Creates an XML sitemap for the phpBB forum

PHP 6Updated: 4 y ago License: Strong Copyleft (GPL-2.0)

PHP 6Updated: 7 y ago License: No License (No License)

Uses Bing search engine to identify (and validate) websites hosted on the same web server

Python 6Updated: 2 y ago License: No License (No License)

PHP 6Updated: 2 y ago License: Strong Copyleft (GPL-2.0)

A lightweight and simple robots.txt parser in node

TypeScript 6Updated: 4 y ago License: Permissive (MIT)

Simple crawler and sitemap generator for Laravel. No headless browser - just a crawler.

PHP 6Updated: 4 y ago License: Permissive (MIT)

Generate chnroute.txt file

Shell 6Updated: 4 y ago License: No License (No License)

A Golang tool to generate sitemap xml for Server Side Rendering apps.

Go 6Updated: 4 y ago License: Permissive (MIT)

Candour's SEO addon for Statamic

PHP 6Updated: 3 y ago License: No License (No License)

Report Lighthouse results from all links in sitemap.xml to Application Insights

JavaScript 6Updated: 2 y ago License: No License (No License)

Sitemap plugin for Winter CMS

PHP 6Updated: 2 y ago License: Permissive (MIT)

This package get, fetch, crawl, sitemap pages recursively and fetch all links in between <loc> tag.

TypeScript 6Updated: 2 y ago License: No License (No License)

Grimbot is a robot that you can customize and teach.

Python 5Updated: 4 y ago License: No License (No License)

Mephisto plugin which implements (Google, MSN, Yahoo ) Sitemap protocol.

Ruby 5Updated: 5 y ago License: Permissive (MIT)

An Ruby Robots.txt parser

Ruby 5Updated: 6 y ago License: Proprietary (Proprietary)

Collect urls within a domain to generate sitemaps. Google sitemaps for example.

Ruby 5Updated: 8 y ago License: No License (No License)

Your friendly gem to produce Google News sitemap compatible xml file

Ruby 5Updated: 4 y ago License: Permissive (MIT)