Remove duplicates of a csv file
Hi,
I am importing a csv and on pcuture 1 is the result.
Now I want to delete all entries which having the same name and Keep only one of them with the latest date. Picture 2
But when I am doing a sort I am only getting the earliest entry Picture 3.
Here the script:
import-csv C:\name8.csv | sort name -Unique
September 1st, 2015 6:50am
Hi Strahle,
well ... sorting with the unique parameter won't work reliably for you. This requires more complex measures:
Import-Csv "C:\name8.csv" | Group Name | %{ $_.Group | Sort Datum -Descending | select -First 1 }
Basically, by grouping on Name, we get a list of all unique names, but those objects have a property named group, where you will find all entries that contained that name. Then for each of those objects we sort those entries by Datum and grab the latest.
Cheers,
Fred
September 1st, 2015 7:58am
And now that you are completely confused how about posting your code?
September 1st, 2015 9:29am
Hi Strahle,
well ... sorting with the unique parameter won't work reliably for you. This requires more complex measures:
Import-Csv "C:\name8.csv" | Group Name | %{ $_.Group | Sort Datum -Descending | select -First 1 }
Basically, by grouping on Name, we get a list of all unique names, but those objects have a property named group, where you will find all entries that contained that name. Then for each of those objects we sort those entries by Datum and grab the latest.
Cheers,
September 1st, 2015 9:32am
Another possible solution:
ht = @{}
Import-Csv "C:\name8.csv" |
foreach {
if ($_.Datum -gt $ht[$_.Name].Datum)
{ $ht[$_.Name] = $_ }
}
$ht.Values
Which is somewhat more complicated code, but has the advantage of not having to accumulate all of the imported objects in memory.
September 1st, 2015 9:47am
Ok one more question:
First setp to Import the csv is working, the result pic 1
$vAuth=Import-CsvC:\name8.csv
In step two I want to Combine some data, result pic 2
$vOut=$vAuth|ForEach-Object{Get-ADUser-Identity$_.name
-Properties*|Select-Object
department,company,SamAccountName
$(
$_.name)}
The Problem is, that I want to have the Information in a raw to Export to csv
-
Edited by
Strahle_fz
16 hours 16 minutes ago
-
September 1st, 2015 10:57am
September 1st, 2015 10:59am
Ok one more question:
First setp to Import the csv is working, the result pic 1
$vAuth=Import-CsvC:\name8.csv
In step two I want to Combine some data, result pic 2
$vOut=$vAuth|ForEach-Object{Get-ADUser-Identity$_.name
-Properties*|Select-Object
department,company,SamAccountName
$(
$_.name)}
The Problem is, that I want to have the Information in a raw to Export to csv
-
Edited by
Strahle_fz
Tuesday, September 01, 2015 2:55 PM
-
September 1st, 2015 2:53pm
Anyone an idea to do that?
September 2nd, 2015 9:48am
Hi Strahle,
well yeah ... that's one of the things you can use Select-Object for:
# Hash Tables used in select-object later on
$Name = @{ n = "Name"; e = { $a.Name } }
$DatumOrg = @{ n = "DatumOrg"; e = { $a.DatumOrg } }
$Datum = @{ n = "Datum"; e = { $a.Datum } }
$vAuth = Import-Csv C:\name8.csv
$vOut = @()
foreach ($a in $vAuth)
{
$vOut += Get-ADUser -Identity $a.name -Properties department, company, SamAccountName | Select-Object $Name, $Datum, $DatumOrg, Department, Company, SamAccountName
}
$vOut
Couldn't bother to set up some test data that will actually work with the script, so you'll need to test it, but it should
work in combining both data sets.
Cheers,
Fred
September 2nd, 2015 10:01am
Hi Strahle,
well yeah ... that's one of the things you can use Select-Object for:
# Hash Tables used in select-object later on
$Name = @{ n = "Name"; e = { $a.Name } }
$DatumOrg = @{ n = "DatumOrg"; e = { $a.DatumOrg } }
$Datum = @{ n = "Datum"; e = { $a.Datum } }
$vAuth = Import-Csv C:\name8.csv
$vOut = @()
foreach ($a in $vAuth)
{
$vOut += Get-ADUser -Identity $a.name -Properties department, company, SamAccountName | Select-Object $Name, $Datum, $DatumOrg, Department, Company, SamAccountName
}
$vOut
Couldn't bother to set up some test data that will actually work with the script, so you'll need to test it, but it should
work in combining both data sets.
Cheers,
September 2nd, 2015 10:17am