Remove duplicates of a csv file

Hi,

I am importing a csv and on pcuture 1 is the result.

Now I want to delete all entries which having the same name and Keep only one of them with the latest date. Picture 2

But when I am doing a sort I am only getting the earliest entry Picture 3.

Here the script:

import-csv C:\name8.csv | sort name -Unique

September 1st, 2015 6:50am

Hi Strahle,

well ... sorting with the unique parameter won't work reliably for you. This requires more complex measures:

Import-Csv "C:\name8.csv" | Group Name | %{ $_.Group | Sort Datum -Descending | select -First 1 }

Basically, by grouping on Name, we get a list of all unique names, but those objects have a property named group, where you will find all entries that contained that name. Then for each of those objects we sort those entries by Datum and grab the latest.

Cheers,
Fred

Free Windows Admin Tool Kit Click here and download it now
September 1st, 2015 7:58am

And now that you are completely confused how about posting your code?
September 1st, 2015 9:29am

Hi Strahle,

well ... sorting with the unique parameter won't work reliably for you. This requires more complex measures:

Import-Csv "C:\name8.csv" | Group Name | %{ $_.Group | Sort Datum -Descending | select -First 1 }

Basically, by grouping on Name, we get a list of all unique names, but those objects have a property named group, where you will find all entries that contained that name. Then for each of those objects we sort those entries by Datum and grab the latest.

Cheers,

Free Windows Admin Tool Kit Click here and download it now
September 1st, 2015 9:32am

Another possible solution:

ht = @{}

Import-Csv "C:\name8.csv" | 
 foreach {
  if ($_.Datum -gt $ht[$_.Name].Datum)
  { $ht[$_.Name] = $_  }
 }

$ht.Values

Which is somewhat more complicated code, but has the advantage of not having to accumulate all of the imported objects in memory.

September 1st, 2015 9:47am

Ok one more question:

First setp to Import the csv is working, the result pic 1

$vAuth=Import-CsvC:\name8.csv

In step two I want to Combine some data, result pic 2

$vOut=$vAuth|ForEach-Object{Get-ADUser-Identity$_.name -Properties*|Select-Object  department,company,SamAccountName


$(

$_.name)}

The Problem is, that I want to have the Information in a raw to Export to csv



Free Windows Admin Tool Kit Click here and download it now
September 1st, 2015 10:57am

September 1st, 2015 10:59am

Ok one more question:

First setp to Import the csv is working, the result pic 1

$vAuth=Import-CsvC:\name8.csv

In step two I want to Combine some data, result pic 2

$vOut=$vAuth|ForEach-Object{Get-ADUser-Identity$_.name -Properties*|Select-Object  department,company,SamAccountName


$(

$_.name)}

The Problem is, that I want to have the Information in a raw to Export to csv



  • Edited by Strahle_fz Tuesday, September 01, 2015 2:55 PM -
Free Windows Admin Tool Kit Click here and download it now
September 1st, 2015 2:53pm

Anyone an idea to do that?
September 2nd, 2015 9:48am

Hi Strahle,

well yeah ... that's one of the things you can use Select-Object for:

# Hash Tables used in select-object later on
$Name = @{ n = "Name"; e = { $a.Name } }
$DatumOrg = @{ n = "DatumOrg"; e = { $a.DatumOrg } }
$Datum = @{ n = "Datum"; e = { $a.Datum } }

$vAuth = Import-Csv C:\name8.csv
$vOut = @()
foreach ($a in $vAuth)
{
	$vOut += Get-ADUser -Identity $a.name -Properties department, company, SamAccountName | Select-Object $Name, $Datum, $DatumOrg, Department, Company, SamAccountName
}
$vOut

Couldn't bother to set up some test data that will actually work with the script, so you'll need to test it, but it  should work in combining both data sets.

Cheers,
Fred

Free Windows Admin Tool Kit Click here and download it now
September 2nd, 2015 10:01am

Hi Strahle,

well yeah ... that's one of the things you can use Select-Object for:

# Hash Tables used in select-object later on
$Name = @{ n = "Name"; e = { $a.Name } }
$DatumOrg = @{ n = "DatumOrg"; e = { $a.DatumOrg } }
$Datum = @{ n = "Datum"; e = { $a.Datum } }

$vAuth = Import-Csv C:\name8.csv
$vOut = @()
foreach ($a in $vAuth)
{
	$vOut += Get-ADUser -Identity $a.name -Properties department, company, SamAccountName | Select-Object $Name, $Datum, $DatumOrg, Department, Company, SamAccountName
}
$vOut

Couldn't bother to set up some test data that will actually work with the script, so you'll need to test it, but it  should work in combining both data sets.

Cheers,

September 2nd, 2015 10:17am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics