Googlebot not respecting http basic auth

I have basic auth set up and it has always worked. Suddenly google started crawling my pages. The auth is still there (I have checked it using different browsers). I am at a loss how it's possible. The user/pass is dead simple to guess from the ur...
more »

2017-01-03 22:01 (0) Answers

How to verify that Google has indexed a page

I've added a new page to my website and currently its being linked to at only one place in the website. The new page is a simple page with some text content. What are some techniques I can use to verify that Googlebot has indexed and crawled said pag...
more »

2016-12-15 17:12 (0) Answers

GoogleBot, SEO, Deep Linking in DB Driven Content

I have a web application that generates dynamic pages from our db. For simplicity, let say the each page generated is about an animal. For every animal(page) you can link to that animals parents, that animals children, that animals owners. And for ea...
more »

2016-11-07 22:11 (0) Answers

AngularJS - Does Routing Harm SEO?

I am sure there are techniques to have a better SEO approach using AngularJS features, (https://prerender.io/js-seo/angularjs-seo-get-your-site-indexed-and-to-the-top-of-the-search-results/). As an example, the source code of a project I am currentl...
more »

2016-10-22 16:10 (0) Answers

Google not crawling links in AngularJS application

I have an AngularJS application that is injected into 3rd party sites. It injects dynamic content into a div on the 3rd party page. Google is successfully indexing this dynamic content but does not appear to be crawling links within the dynamic conte...
more »

2016-10-13 00:10 (2) Answers

Serve different page to google bot crawler

I have an SPA with a lot of images in it. I want to expose those images to search engines. So I want to create "special" pages that will only be seen by the bot. The pages will contain metadata about the images. Is it possible to make googlebot cra...
more »

2016-05-25 13:05 (2) Answers

Single page application SEO and canonical link

I have an SPA that handles many digital assets. I want all these assets to be indexed by search engines. To do this I've created 2 new pages in my site. These pages are only shown to bots crawling my site. The first page contains a paginated list of ...
more »

2016-05-24 10:05 (0) Answers