Gocrawler

Simple crawler based on the sitemap of a URL

Run it with Docker

docker run danielnaveda/gocrawler -domain "https://www.a-domain-with-a-standard-sitemap.com"

docker run -v <your local full path>:/go/temp-files danielnaveda/gocrawler -domain "https://www.a-domain-with-a-standard-sitemap.com"

Name		Name	Last commit message	Last commit date
Latest commit History 45 Commits
app		app
conf		conf
files		files
sitemap		sitemap
worker		worker
.gitignore		.gitignore
.travis.yml		.travis.yml
Dockerfile		Dockerfile
LICENSE		LICENSE
conf.dist.yaml		conf.dist.yaml
main.go		main.go
readme.md		readme.md