Googlebot making POST requests

I am seeing lots of googlebot requests with a POST method. I verified that requester ip is from google. There is no ajax content on the page. It does POST requests to home page, content pages. etc. In this case, is there a way to tell google...
more »

2017-08-21 00:08 (0) Answers

Google robots and no-copy content

I was wondering about Google robots algorithm an how they read <div> content, where I have this configuration in CSS: -webkit-touch-callout: none; /* iOS Safari */ -webkit-user-select: none; /* Safari */ -khtml-user-select: none; /* Konqueror ...
more »

2017-06-25 19:06 (0) Answers

Module preventing GoogleBot from reading page

I've a React+Rails application in which I'm using the React-rte module to allow users to send a stylized message. The problem is that if I include the module in my code then Fetch as Google returns an empty page for my application. Simply removing t...
more »

2017-05-03 22:05 (0) Answers

vary (Dynamic serving) user-agent example

Google suggest us to use Vary header to deliver dynamic content for mobile and desktop pages. Anyone who know how to do it with simple html or php help will be appreciated For example i have two pages index.html index_mobile.html How can i serv...
more »

2017-02-13 06:02 (0) Answers

Googlebot not respecting http basic auth

I have basic auth set up and it has always worked. Suddenly google started crawling my pages. The auth is still there (I have checked it using different browsers). I am at a loss how it's possible. The user/pass is dead simple to guess from the ur...
more »

2017-01-03 22:01 (0) Answers

How to verify that Google has indexed a page

I've added a new page to my website and currently its being linked to at only one place in the website. The new page is a simple page with some text content. What are some techniques I can use to verify that Googlebot has indexed and crawled said pag...
more »

2016-12-15 17:12 (0) Answers

GoogleBot, SEO, Deep Linking in DB Driven Content

I have a web application that generates dynamic pages from our db. For simplicity, let say the each page generated is about an animal. For every animal(page) you can link to that animals parents, that animals children, that animals owners. And for ea...
more »

2016-11-07 22:11 (0) Answers

AngularJS - Does Routing Harm SEO?

I am sure there are techniques to have a better SEO approach using AngularJS features, ( As an example, the source code of a project I am currentl...
more »

2016-10-22 16:10 (0) Answers

Google not crawling links in AngularJS application

I have an AngularJS application that is injected into 3rd party sites. It injects dynamic content into a div on the 3rd party page. Google is successfully indexing this dynamic content but does not appear to be crawling links within the dynamic conte...
more »

2016-10-13 00:10 (2) Answers

Serve different page to google bot crawler

I have an SPA with a lot of images in it. I want to expose those images to search engines. So I want to create "special" pages that will only be seen by the bot. The pages will contain metadata about the images. Is it possible to make googlebot cra...
more »

2016-05-25 13:05 (2) Answers

Single page application SEO and canonical link

I have an SPA that handles many digital assets. I want all these assets to be indexed by search engines. To do this I've created 2 new pages in my site. These pages are only shown to bots crawling my site. The first page contains a paginated list of ...
more »

2016-05-24 10:05 (0) Answers