# ============================================ # MAGENTO 2 SPECIFIC BLOCKS # ============================================ User-agent: * # Admin and system directories Disallow: /admin/ Disallow: /downloader/ Disallow: /errors/ Disallow: /includes/ Disallow: /lib/ Disallow: /phpserver/ Disallow: /pkginfo/ Disallow: /setup/ Disallow: /update/ Disallow: /var/ Disallow: /dev/ # API and technical endpoints Disallow: /api/ Disallow: /rest/ Disallow: /soap/ Disallow: /graphql/ Disallow: /cron.php Disallow: /cron.sh # Customer private areas Disallow: /customer/ Disallow: /wishlist/ Disallow: /checkout/ Disallow: /onepage/ Disallow: /paypal/ # Cart and checkout Disallow: /cart/ Disallow: /checkout/cart/ Disallow: /multishipping/ # Search and duplicate content Disallow: /catalogsearch/ Disallow: /catalog/product_compare/ Disallow: /catalog/category/view/ Disallow: /catalog/product/view/ # System files and pub Disallow: /*.php$ Disallow: /pub/ # URL parameters (duplicate content) Disallow: /? Disallow: /?product_list_limit= Disallow: /?product_list_mode= Disallow: /?product_list_order= Disallow: /?product_list_dir= Disallow: /?SID= Disallow: /?sid= Disallow: /*___store= Disallow: /*___from_store= Disallow: /?dir= Disallow: /?order= Disallow: /?limit= Disallow: /?mode= Disallow: /*?p= # Private media Disallow: /media/customer/ Disallow: /media/downloadable/ # ============================================ # ALLOW IMPORTANT ASSETS # ============================================ Allow: /media/ Allow: /static/ Allow: /pub/static/ Allow: /pub/media/ Allow: /media/catalog/ Allow: /.jpg$ Allow: /.jpeg$ Allow: /.png$ Allow: /.gif$ Allow: /.webp$ Allow: /.svg$ Allow: /.css$ Allow: /.js$ # Sitemap Sitemap: https://www.inorganicventures.com/sitemap.xml