Skip to content

Build URL graph

  • Parse the downloaded HTMLs
  • Report found URLs
  • Build a directed graph of URLs
  • Do BFS on it
  • Stay within original domain