ddatacommons.org

The URL responds, but the body does not open as an XML urlset or sitemapindex.

Consequence. A sitemap that is not valid XML is discarded. A common cause is the URL resolving to an HTML error page or a redirect to the home page.

If that is not intended. Confirm the URL serves XML beginning with <urlset> or <sitemapindex>, and that it is served as application/xml or text/xml.

• https://datacommons.org/sitemap/AdministrativeArea3.0.txtL1HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)
• https://datacommons.org/sitemap/AdministrativeArea3.1.txtL2HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)
• https://datacommons.org/sitemap/AdministrativeArea3.2.txtL3HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)
• https://datacommons.org/sitemap/AdministrativeArea1.0.txtL4HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)
• https://datacommons.org/sitemap/CongressionalDistrict.0.txtL5HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)
• https://datacommons.org/sitemap/City.1.txtL6HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)
• https://datacommons.org/sitemap/County.0.txtL7HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)
• https://datacommons.org/sitemap/City.0.txtL8HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)
• https://datacommons.org/sitemap/City.2.txtL9HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)
• https://datacommons.org/sitemap/CensusCoreBasedStatisticalArea.0.txtL10HTTP 200, but the body does not open as XML (Content-Type: text/plain; charset=utf-8)

The file declares no User-agent groups, so every crawler may fetch every URL.

Consequence. Nothing is restricted. That is a valid and common configuration, but it also means no crawl budget is being steered away from low-value pages.

If that is not intended. If some sections should not be crawled — faceted search, internal endpoints, duplicate parameter URLs — declare them with Disallow rules.

All 6 checked search engines are allowed at the site root.

All 4 checked AI answer engines are allowed at the site root.

All 3 checked user-triggered AI fetches are allowed at the site root.

All 5 checked social link previews are allowed at the site root.

All 6 checked AI training crawlers are allowed at the site root.

The page sends no robots meta tag and no X-Robots-Tag header.

These sitemaps were not seen: each was blocked for our bot, left out by a limit, or did not answer in time. Each line says which.

Consequence. Nothing is known about whether they work. They are not counted against the file.

• https://datacommons.org/sitemap/StateComponent.0.txtL11Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/AdministrativeArea2.0.txtL12Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/AdministrativeArea2.1.txtL13Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/PriorityPlaces.0.txtL14Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/AdministrativeArea4.3.txtL15Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/AdministrativeArea4.2.txtL16Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/Continent.0.txtL17Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/CommutingZone.0.txtL18Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/AdministrativeArea4.0.txtL19Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/CensusCountyDivision.0.txtL20Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/AdministrativeArea4.1.txtL21Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/State.0.txtL22Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/EurostatNUTS3.0.txtL23Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/Country.0.txtL24Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/stat_vars.txtL25Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/CensusZipCodeTabulationArea.0.txtL26Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/EurostatNUTS1.0.txtL27Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/SchoolDistrict.0.txtL28Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/CensusTract.2.txtL29Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/CensusTract.1.txtL30Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/EurostatNUTS2.0.txtL31Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/CensusTract.0.txtL32Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/HighSchoolDistrict.0.txtL33Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay
• https://datacommons.org/sitemap/ElementarySchoolDistrict.0.txtL34Not fetched: only the first few sitemaps are fetched, within a time limit and the site's Crawl-delay

Every line parses as a directive crawlers understand.

Comfortably within the 500 KB that crawlers will read.