Script to download file from a website on a weekly basis

Hello,

Not sure if this is possible via script or require a specific software.  Each week one of my user must download file from link below in zip format and would like to know if possible to create as script for doing so.

The script will work as follow:

1- Open link below.

2- download latest file which has name based on date of week (file is generated each Monday)

3. A prompt will appear for password..enter password

4. Save file to a directory on a server such as C:\WeeklyData

https://dea.ntis.gov/zips/weekly.html

Thanks...B

February 27th, 2014 4:58pm

The script at the first execution will dowload all the files, in the subsequent execution only new files will downloaded.

Change the ReportFolder and fill  $Username and  $Password  correctly

cls #<tr valign="top"><td><a href="../deadata/zipweekly/WA131111.zip">WA131111.zip</a> </td><td> 69.1 MB </td><td> 2013/11/11 </td><td align="center"> Weekly </td><td align="center"> 1512416</td> $ReportFolder="c:\temp" $DebugPreference = "SilentlyContinue" $wc = new-object System.Net.WebClient $userAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2;)" $wc.Headers.Add("user-agent", $userAgent) $MainUrl = "https://dea.ntis.gov/zips/weekly.html" $ReportBlock=".*\.zip" $ZipUrl="https://dea.ntis.gov/deadata/zipweekly"
#$usr=read-host
#$pwd=read-host -assecurestring $Username = "" $Password = "" cd $ReportFolder $ReportPage = $wc.DownloadString($MainUrl) $matches=[string] $ReportPage |select-string -pattern "$ReportBlock" -AllMatches $wc.Credentials = New-Object System.Net.Networkcredential($Username, $Password) foreach($match in $matches.Matches) { $txt = ( ($match.Value -replace "</A><br>", "`r`n") -replace "<[^>]*?>","") foreach($lines in $txt.Split("`r`n")){ $line=$lines|select-string -NotMatch -Pattern "To Parent|^$|&lt;dir&gt;" if ($line -ne $null){ if (!(Test-Path "$ReportFolder\$line")) { Try { $wc.DownloadFile("$ZipUrl/$line","$ReportLocalDir\$_" ) } catch { echo "An error occurred: $line" } } } } }


Bye Gas

Free Windows Admin Tool Kit Click here and download it now
February 27th, 2014 7:09pm

The script at the first execution will dowload all the files, in the subsequent execution only new files will downloaded.

Change the ReportFolder and fill  $Username and  $Password  correctly

cls #<tr valign="top"><td><a href="../deadata/zipweekly/WA131111.zip">WA131111.zip</a> </td><td> 69.1 MB </td><td> 2013/11/11 </td><td align="center"> Weekly </td><td align="center"> 1512416</td> $ReportFolder="c:\temp" $DebugPreference = "SilentlyContinue" $wc = new-object System.Net.WebClient $userAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2;)" $wc.Headers.Add("user-agent", $userAgent) $MainUrl = "https://dea.ntis.gov/zips/weekly.html" $ReportBlock=".*\.zip" $ZipUrl="https://dea.ntis.gov/deadata/zipweekly"
#$usr=read-host
#$pwd=read-host -assecurestring $Username = "" $Password = "" cd $ReportFolder $ReportPage = $wc.DownloadString($MainUrl) $matches=[string] $ReportPage |select-string -pattern "$ReportBlock" -AllMatches $wc.Credentials = New-Object System.Net.Networkcredential($Username, $Password) foreach($match in $matches.Matches) { $txt = ( ($match.Value -replace "</A><br>", "`r`n") -replace "<[^>]*?>","") foreach($lines in $txt.Split("`r`n")){ $line=$lines|select-string -NotMatch -Pattern "To Parent|^$|&lt;dir&gt;" if ($line -ne $null){ if (!(Test-Path "$ReportFolder\$line")) { Try { $wc.DownloadFile("$ZipUrl/$line","$ReportLocalDir\$_" ) } catch { echo "An error occurred: $line" } } } } }


B

March 4th, 2014 5:35pm

Retry with this
cls
#<tr valign="top"><td><a href="../deadata/zipweekly/WA131111.zip">WA131111.zip</a> </td><td> 69.1 MB </td><td> 2013/11/11 </td><td align="center"> Weekly </td><td align="center"> 1512416</td>  
 
$ReportFolder="C:\Data\DEAWeekly"
$DebugPreference = "SilentlyContinue"
$wc = new-object System.Net.WebClient
$userAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2;)"
$wc.Headers.Add("user-agent", $userAgent)
$MainUrl = "https://dea.ntis.gov/zips/weekly.html"
$ReportBlock=".*\.zip"
$ZipUrl="https://dea.ntis.gov/deadata/zipweekly"
#$usr=read-host 
#$pwd=read-host -assecurestring
$Username = "kkk"
$Password = "kk"

cd $ReportFolder
$ReportPage = $wc.DownloadString($MainUrl)
$matches=[string] $ReportPage |select-string -pattern  "$ReportBlock" -AllMatches
$wc.Credentials = New-Object System.Net.Networkcredential($Username, $Password)

foreach($match in $matches.Matches) {	
   	$txt = ( ($match.Value  -replace "</A><br>", "`r`n") -replace  "<[^>]*?>","")
   	foreach($lines in $txt.Split("`r`n")){
      $line=$lines|select-string  -NotMatch -Pattern "To Parent|^$|&lt;dir&gt;"
      if ($line -ne $null){
		 if (!(Test-Path "$ReportFolder\$line"))  { 
		 	Try {	
           		$wc.DownloadFile("$ZipUrl/$line","$ReportFolder\$line" )
       		}
	    	catch [System.Net.WebException]  {
				echo "An error occurred: $ZipUrl/$line`r`n$ReportLocalDir\$_"
				if ($_.Exception.InnerException) {"$($_.Exception.InnerException.Message)"}
			}
		 }
      }
   }
}

Free Windows Admin Tool Kit Click here and download it now
March 4th, 2014 6:36pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics