wget utility in Linux

Introductions:

It is the utility which is no-interactive network downloader. It is used to download the files from the server even when the user is not logged on the system. It can work in the background without hinder the current process.

It is a free utility which is non-interactive download of files through the web provided by GNU. It support the both HTTP and HTTPS protocols. It also support FTP and HTTP proxies as well.

wget provide a number of options allowing you to download multiple files, will resume downloads, limit the bandwidths, recursive downloads and mirror a website.

Non-interactive means work in the background, while the user is not logged in. It also allow to start a retrieval and disconnect from the system, let wget finish the work. Most of the web browsers is require a constant user presence for a good hinder and transfer the data.

Lets use and check the version of wget.

How to install wget in the system

It is the package which is pre-installed in the most Linux distributions today. To check if wget package is installed in on linux system or not just open terminal and type wget

and press enter.

If wget is not installed, we can easily install it using the package manager of distro in desktop system or can install using terminal of operating system.

Syntax :

wget [option] [URL]

Example :

1. To simply download a webpage:

wget http://www.example.com

2. To download the file in background

wget -b http://www.example.com/samplepage.php

3. Resume a partially downloaded file

wget -c http://example.com/samplefile.tar.gz

4. To try a given number of times

wget –tries=10 http://example.com/samplefile.tar.gz

5. downloading the standard output.

wget -q -O – “http://sample.com/latest.tar.gz” | tar -xzf – -C /var/www

Options :

1. -v : This is used to display the version of the wget available on your system.

Syntax

$wget -v

2. -h : This is used to print a help message displaying all the possible options of command line are available with wget.

Syntax

$wget -h [URL]

3. -o log file : This option is used to direct all the messages generated by the system to the log file specified by the option and when the process is completed all the messages thus generated are available in the log file. If no log file has been specified then the output messages are redirected to the default log file i.e. wget -log

Syntax

$wget -o logfile [URL]

4. -b / –background : This option is used to send a process to the background as soon as the process has started so that other processes can be carried out. If no output file is specified via the -o option, output is redirected to wget-log by default.

Syntax

$wget -b [URL]

5. -a : This option is used to append the output messages to the current output log file without overwriting the file as in -o option the output log file is overwritten but by using this option the log of the previous command is saved and the current log is written after that of the previous ones.

Syntax

$wget -a logfile [URL]

6. -i : This option is used to read URLs from file. If -i is specified as file, URLs are read from the standard input.If this function is used, no URLs need be present on the command line. If there are URLs both on the command line and in an input file, those on the command lines will be the first ones to be retrieved. The file need not be an HTML document if the URLs are just listed sequentially.

Syntax

$wget -i input-file $wget -i input-file [URL]

7. -t number / –tries=number : This option is used to set number of retries to a specified number of times. Specify 0 or inf for infinite retrying. The default is to retry 20 times, with the exception of fatal errors like connection refused or link not found, which are not retried once the error has occurred.

Syntax

$wget -t number [URL]

8. -c : This option is used to resume a partially downloaded file if the resume capability of the file is yes otherwise the downloading of the file cannot be resume if the resume capability of the given file is no or not specified.

Syntax

$wget -c [URL]

9. -w : This option is used to set the system to wait the specified number of seconds between the retrievals. Use of this option is recommended, as it lightens the server load by making the requests less frequent. Instead of in seconds, the time can be specified in minutes using the m suffix, in hours using h suffix, or in days using d suffix. Specifying a large value for this option is useful if the network or the destination host is down, so that wget can wait long enough to reasonably expect the network error to be fixed before the retry.

Syntax

$wget -w number in seconds [URL]

10. -r : this option is used to turn on the recursive retrieving of the link specified in case of fatal errors also. This option is a recursive call to the given link in the command line.

Syntax

$wget -r [URL]

We have seen the use of wget utility with examples of the syntax and tags and also how wget command work in system.