יום רביעי, 10 באוקטובר 2018

Can bot spy on IoT bot?

Today I was trying to compare the anti robot rules between the major IoT search engine.

Starting from Shodan later Censys and ZoomEye and finalizing with Fofa. 

 Shodan



User-Agent: Twitterbot
Allow: /host/

# Every bot that might possibly read and respect this file.
User-agent: *
Crawl-delay: 10
Disallow: /search*
Disallow: /host/
Disallow: /report/

 Censys






User-agent: *
Disallow: /ipv4
Allow:    /ipv4/
Disallow: /ipv4/metadata
Disallow: /ipv4/*/
Disallow: /certificates
Allow:    /certificates/
Disallow: /certificates/metadata
Disallow: /certificates/*/
Disallow: /domain
Allow:    /domain/
Disallow: /domain/metadata
Disallow: /domain/*/
Disallow: /data/scansio
Disallow: /login
Disallow: /logout
Disallow: /account



ZoomEye






As you can see in ZoomEye there are no rules.


Fofa



# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
#
# To ban all spiders from the entire site uncomment the next two lines:
# User-agent: *
# Disallow: /


המצור הדיגיטלי של טינדר על רצועת עזה והאזורים שמעבר לקו הירוק, ומה הקשר לצפון קוריאה?

For my English reader: I've found that Passport  feature which is part of Tinder Plus services , is not supported in Gaza and behind ...