Nmap http-fetch NSE Script


This page contains detailed information about how to use the http-fetch NSE script. For list of all NSE scripts, visit the Nmap NSE Library.

Select:
Overview
Error Messages

Script Overview

Script source code: https://github.com/nmap/nmap/tree/master/scripts/http-fetch.nse
Script categories: safe
Target service / protocol: http, https
Target network port(s): 80, 443, 631, 7080, 8080, 8443, 8088, 5800, 3872, 8180, 8000
List of CVEs: -

Script Description

The script is used to fetch files from servers.

The script supports three different use cases:

  • The paths argument isn't provided, the script spiders the host and downloads files in their respective folders relative to the one provided using "destination".
  • The paths argument(a single item or list) is provided and the path starts with "/", the script tries to fetch the path relative to the url provided via the argument "url".
  • The paths argument(a single item or list) is provided and the path doesn't start with "/". Then the script spiders the host and tries to find files which contain the path(now treated as a pattern).

Http-fetch NSE Script Arguments

This is a full list of arguments supported by the http-fetch.nse script:

http-fetch.destination

The full path of the directory to save the file(s) to preferably with the trailing slash.

http-fetch.files

The name of the file(s) to be fetched.

http-fetch.maxdepth

The maximum amount of directories beneath the initial url to spider. A negative value disables the limit. (default: 3)

http-fetch.maxpagecount

The maximum amount of pages to fetch.

http-fetch.noblacklist

By default files like jpg, rar, png are blocked. To fetch such files set noblacklist to true.

http-fetch.paths

A list of paths to fetch. If relative, then the site will be spidered to find matching filenames. Otherwise, they will be fetched relative to the url script-arg.

http-fetch.url

The base URL to start fetching. Default: "/"

http-fetch.withindomain

If set to true then the crawling would be restricted to the domain provided by the user.

http-fetch.withinhost

The default behavior is to fetch files from the same host. Set to False to do otherwise.

smbdomain

The domain to log in with. If you aren't in a domain environment, then anything will (should?) be accepted by the server.

smbhash

A password hash to use when logging in. This is given as a single hex string (32 characters) or a pair of hex strings (both 32 characters, optionally separated by a single character). These hashes are the LanMan or NTLM hash of the user's password, and are stored on disk or in memory. They can be retrieved from memory using the fgdump or pwdump tools.

smbnoguest

Use to disable usage of the 'guest' account.

smbpassword

The password to connect with. Be cautious with this, since some servers will lock accounts if the incorrect password is given. Although it's rare that the Administrator account can be locked out, in the off chance that it can, you could get yourself in trouble. To use a blank password, leave this parameter off altogether.

smbtype

The type of SMB authentication to use. These are the possible options:

  • v1: Sends LMv1 and NTLMv1.
  • LMv1: Sends LMv1 only.
  • NTLMv1: Sends NTLMv1 only (default).
  • v2: Sends LMv2 and NTLMv2.
  • LMv2: Sends LMv2 only.
  • NTLMv2: Doesn't exist; the protocol doesn't support NTLMv2 alone. The default, NTLMv1, is a pretty decent compromise between security and compatibility. If you are paranoid, you might want to use v2 or lmv2 for this. (Actually, if you're paranoid, you should be avoiding this protocol altogether!). If you're using an extremely old system, you might need to set this to v1 or lm, which are less secure but more compatible. For information, see smbauth.lua.

smbusername

The SMB username to log in with. The forms "DOMAIN\username" and "username@DOMAIN" are not understood. To set a domain, use the smbdomain argument.

slaxml.debug

Debug level at which default callbacks will print detailed parsing info. Default: 3

http.host

The value to use in the Host header of all requests unless otherwise set. By default, the Host header uses the output of stdnse.get_hostname().

http.max-body-size

Limit the received body to specific number of bytes. An oversized body results in an error unless script argument http.truncated-ok or request option truncated_ok is set to true. The default is 2097152 (2MB). Use value -1 to disable the limit altogether. This argument can be overridden case-by-case with request option max_body_size.

http.max-cache-size

The maximum memory size (in bytes) of the cache.

http.max-pipeline

If set, it represents the number of outstanding HTTP requests that should be sent together in a single burst. Defaults to http.pipeline (if set), or to what function get_pipeline_limit returns.

http.pipeline

If set, it represents the number of HTTP requests that'll be sent on one connection. This can be set low to make debugging easier, or it can be set high to test how a server reacts (its chosen max is ignored).

http.truncated-ok

Do not treat oversized body as error. (Use response object flag truncated to check if the returned body has been truncated.) This argument can be overridden case-by-case with request option truncated_ok.

http.useragent

The value of the User-Agent header field sent with requests. By default it is "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)". A value of the empty string disables sending the User-Agent header field.

- - -
To use these script arguments, add them to the Nmap command line using the --script-args arg1=value,[arg2=value,..] syntax. For example:

nmap --script=http-fetch --script-args http-fetch.destination=value,http-fetch.files=value <target>

Http-fetch NSE Script Example Usage

Here's an example of how to use the http-fetch.nse script:

nmap --script http-fetch --script-args destination=/tmp/mirror <target>

nmap --script http-fetch --script-args 'paths={/robots.txt,/favicon.ico}' <target>

nmap --script http-fetch --script-args 'paths=.html' <target>

nmap --script http-fetch --script-args 'url=/images,paths={.jpg,.png,.gif}' <target>

Http-fetch NSE Script Example Output

Here's a sample output from the http-fetch.nse script:

| http-fetch:
|   Successfully Downloaded:
|     http://scanme.nmap.org:80/ as /tmp/mirror/45.33.32.156/80/index.html
|_    http://scanme.nmap.org/shared/css/insecdb.css as /tmp/mirror/45.33.32.156/80/shared/css/insecdb.css

Http-fetch NSE Script Example XML Output

Here's a sample XML output from the http-fetch.nse script produced by providing the -oX <file> Nmap option:

 <table key="Successfully Downloaded">
   <elem>http://scanme.nmap.org:80/ as /tmp/mirror/45.33.32.156/80/index.html</elem>
   <elem>http://scanme.nmap.org/shared/css/insecdb.css as /tmp/mirror/45.33.32.156/80/shared/css/insecdb.css</elem>
 </table>
 <elem key="result">Successfully Downloaded Everything At: /tmp/mirror/45.33.32.156/80/</elem>

Author

  • Gyanendra Mishra

References

See Also

Visit Nmap NSE Library for more scripts.

The http-fetch.nse script may fail with the following error messages. Check for the possible causes by using the code snippets highlighted below found in the script source code. This can often times help in identifying the root cause of the problem.

Error encountered in writing file.. %s

Here is a relevant code snippet related to the "Error encountered in writing file.. %s" error message:

117:	    stdnse.debug1("Saving to ...%s",file_path)
118:	    file:write(content)
119:	    file:close()
120:	    return true, file_path
121:	  else
122:	    stdnse.debug1("Error encountered in  writing file.. %s",err)
123:	    return false, err
124:	  end
125:	end
126:	
127:	local function fetch_recursively(host, port, url, destination, patterns, output)

ERROR'] = output['ERROR

Here is a relevant code snippet related to the "ERROR'] = output['ERROR" error message:

146:	          if status then
147:	            output['Matches'] = output['Matches'] or {}
148:	            output['Matches'][pattern] = output['Matches'][pattern] or {}
149:	            table.insert(output['Matches'][pattern], string.format("%s as %s",r.url:getFile()),err_message)
150:	          else
151:	            output['ERROR'] = output['ERROR'] or {}
152:	            output['ERROR'][url_string] = err_message
153:	          end
154:	          break
155:	        end
156:	      end

ERROR'] = output['ERROR

Here is a relevant code snippet related to the "ERROR'] = output['ERROR" error message:

159:	      local stat, path_or_err = save_file(body, nil, destination, r.url)
160:	      if stat then
161:	        output['Successfully Downloaded'] = output['Successfully Downloaded'] or {}
162:	        table.insert(output['Successfully Downloaded'], string.format("%s as %s", url_string, path_or_err))
163:	      else
164:	        output['ERROR'] = output['ERROR'] or {}
165:	        output['ERROR'][url_string] = path_or_err
166:	      end
167:	    else
168:	      if not r.response.body then
169:	        stdnse.debug1("No Body For: %s",url_string)

ERROR'] = output['ERROR

Here is a relevant code snippet related to the "ERROR'] = output['ERROR" error message:

185:	    local status, err_message = save_file(response.body, save_as, destination)
186:	    if status then
187:	      output['Successfully Downloaded'] = output['Successfully Downloaded'] or {}
188:	      table.insert(output['Successfully Downloaded'], string.format("%s as %s", path, save_as))
189:	    else
190:	      output['ERROR'] = output['ERROR'] or {}
191:	      output['ERROR'][path] = err_message
192:	    end
193:	  else
194:	    stdnse.debug1("%s doesn't exist on server at %s.", path, url)
195:	  end

Please enter the complete path of the directory to save data in.

Here is a relevant code snippet related to the "Please enter the complete path of the directory to save data in." error message:

203:	
204:	  local output = stdnse.output_table()
205:	  local patterns = {}
206:	
207:	  if not destination then
208:	    output.ERROR = "Please enter the complete path of the directory to save data in."
209:	    return output, output.ERROR
210:	  end
211:	
212:	  local sub_directory = tostring(host.ip) .. SEPARATOR ..  tostring(port.number) .. SEPARATOR
213:

Version

This page has been created based on Nmap version 7.92.

Go back to menu.