Sponsors
Sponsor Products
recover deleted text files from linux xfs
posted by Greg Whynott  on May 3, 2017, 12:30 p.m. (3 years, 2 months, 11 days ago)
6 Responses     0 Plus One's     0 Comments  

I wrote a script some time back on a machine which was rsync'n to a partner using a password. The machine developed problems and the help desk guys re-imaged it upon request.

naturally the next day we got a ticket saying lastnights rsync didn't happen. Whoops... forgot to backup the script and the machine isn't in the backup loop.. Not wanting to admit anything or call the partner to ask them what the password is again, I decided to dd the file system and search for it.

At about the same time I finished re-writing the script, my grepper script found the old one and the password file! Get to avoid asking for a password and being 'that guy'. :)


The short here is if you delete something from XFS/EXT, or reformat the drive, there still is hope. I found the password file and the password days later on a live disk, using included GNU tools on the system (grep and dd).


Crez^D<96><90>^F^@^C^Drepo^D<96><90>^F^@^A^Coot^D<96><90>^F^@^A^Dsync^D<96><90>^F^@^E^Hexcludes^D<96><90>^F^@^E^Dpass^D<96><90>^F^@^@^Dsdir^D<96><90>^F^@^A^Ah^D<96><90>^F^@^A^Dtats^D<96><90>^F^@^B^Dudio^D<96><90>^F^@^@^Dtest^D<96><90>^F^@^A^Foolbox^D<96><90>^F^@^@^Duser^D<96><90>^F^@^@^@^F^B^@^@^@^@1^@MAR&@,^]^M^C<94>^@^A^@>^@^@>^C<94>^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<86>Q<86><90>^VG^@^@^@^@^@^@^@^@^@?^C^@^@<8b>W^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^C^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^
@^@^@^@^@^@application/x-shellscript^A<86>#! /bin/sh
--
CMD="rsync -avz --links --progress --stats --delete --force --chown=greg:greg --exclude-from /root/.rsyncexcludes "
USER=rezrepo
PASSFILE=/root/.rsyncpass
AUTH="--password-file /root/.rsyncpass"


$CMD $AUTH $SDIR rsync://$USER@$DIP/$DEST


^A<86>^A<88>^V^A<86>^@^@^@l^@^@^@^@^@^@^@^



here is the command I used:

grep -i -a -B10 'rsyncpass' /dev/mapper/centos7-root > findstuff.txt




-g



Thread Tags:
  discuss-at-studiosysadmins 

Response from Dan Young @ May 8, 2017, 3 p.m.
**clickhole warning**

On Mon, May 8, 2017 at 2:07 PM, Jean-Francois Panisset <panisset@gmail.com> wrote:
I could watch this for hours...

https://www.ssiworld.com/en/watch_it_shred


On Sun, May 7, 2017 at 3:32 PM, Dan Mons <dmons@cuttingedge.com.au> wrote:
On 8 May 2017 at 05:47, Ben De Luca<bdeluca@gmail.com>wrote:
https://linux.die.net/man/1/shred

Interesting security issue with shred: certain brands of drives (particularly SSDs and their weird firmware) move writes to physically different locations, even if the request specifically was to overwrite a given sector. (Generally not an issue on single platter drives, however).

What this means is shred's overwrite with zero/random data returns successful, but actually happened elsewhere within the physical layout of the cells. Supposedly removed data can sometimes be later retrieved via very simple tools as a result. Add to that the fact that many SSDs keep extra cells as spare for wear levelling, and even a full random write to the entire logical drive as presented by the firmware doesn't touch 100% of the physical storage.
Then you've also got swap, which frequently contains a lot of interesting things, even after reboot.
All of these are now a big part of why tools like "DBAN" and other software level wiping tools are no longer supported by either the military or big finance (SOX/ISO27001/etc). These days a degauss and a trip to the industrial shredder are their methods for dealing with that side of things. Not much help at the file level, however.
Specific to SSDs, resetting the cells is considered "secure enough" for most people. But again, drive level, not file level, which is a bummer.https://wiki.archlinux.org/index.php/Solid_State_Drives/Memory_cell_clearing

Another vote for Open Channel SSDs here. No more pesky vendor firmware getting between me and the data I write, please.
-Dan



Thisemailis confidential and solely for the use of the intended recipient.If you have received thisemailin error please notify the author and delete it immediately. Thisemailis not to be distributed without the author's written consent.Unauthorised forwarding, printing, copying or use is strictly prohibited and may be a breach of copyright. Any views expressed in thisemailare those of the individual sender unless specifically stated to be the views ofCutting EdgePost Pty Ltd (Cutting Edge). Although thisemailhas been sent in the belief that it is virus-free, it is the responsibility of the recipient to ensure that it is virus free. No responsibility is accepted byCutting Edgefor any loss or damage arising in any way from receipt or use of thisemail. Thisemailmay contain legally privileged information and privilege is not waived if you have received thisemailin error.



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



--

Framestore
Dan Young Lead Systems Engineer
London New York Los Angeles Chicago Montral
T+1 212 775 0600
135 Spring Street, New York NY 10012
Twitter Facebook framestore.com

0 Plus One's     0 Comments  
   

Response from Jean-Francois Panisset @ May 8, 2017, 2:10 p.m.
I could watch this for hours...

https://www.ssiworld.com/en/watch_it_shred


On Sun, May 7, 2017 at 3:32 PM, Dan Mons <dmons@cuttingedge.com.au> wrote:
On 8 May 2017 at 05:47, Ben De Luca<bdeluca@gmail.com>wrote:
https://linux.die.net/man/1/shred

Interesting security issue with shred: certain brands of drives (particularly SSDs and their weird firmware) move writes to physically different locations, even if the request specifically was to overwrite a given sector. (Generally not an issue on single platter drives, however).

What this means is shred's overwrite with zero/random data returns successful, but actually happened elsewhere within the physical layout of the cells. Supposedly removed data can sometimes be later retrieved via very simple tools as a result. Add to that the fact that many SSDs keep extra cells as spare for wear levelling, and even a full random write to the entire logical drive as presented by the firmware doesn't touch 100% of the physical storage.
Then you've also got swap, which frequently contains a lot of interesting things, even after reboot.
All of these are now a big part of why tools like "DBAN" and other software level wiping tools are no longer supported by either the military or big finance (SOX/ISO27001/etc). These days a degauss and a trip to the industrial shredder are their methods for dealing with that side of things. Not much help at the file level, however.
Specific to SSDs, resetting the cells is considered "secure enough" for most people. But again, drive level, not file level, which is a bummer.https://wiki.archlinux.org/index.php/Solid_State_Drives/Memory_cell_clearing

Another vote for Open Channel SSDs here. No more pesky vendor firmware getting between me and the data I write, please.
-Dan



Thisemailis confidential and solely for the use of the intended recipient.If you have received thisemailin error please notify the author and delete it immediately. Thisemailis not to be distributed without the author's written consent.Unauthorised forwarding, printing, copying or use is strictly prohibited and may be a breach of copyright. Any views expressed in thisemailare those of the individual sender unless specifically stated to be the views ofCutting EdgePost Pty Ltd (Cutting Edge). Although thisemailhas been sent in the belief that it is virus-free, it is the responsibility of the recipient to ensure that it is virus free. No responsibility is accepted byCutting Edgefor any loss or damage arising in any way from receipt or use of thisemail. Thisemailmay contain legally privileged information and privilege is not waived if you have received thisemailin error.



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments  
   

Response from Ben De Luca @ May 7, 2017, 3:50 p.m.
https://linux.die.net/man/1/shred

On 4 May 2017 at 00:21, greg whynott <greg.whynott@gmail.com> wrote:
XFS calls home? *gasp!*

On Wed, May 3, 2017 at 3:36 PM, Greg Dickie <greg@justaguy.ca> wrote:
Just ask the NSA.
Greg
On Wed, May 3, 2017 at 12:26 PM, greg whynott <greg.whynott@gmail.com> wrote:

I wrote a script some time back on a machine which was rsync'n to a partner using a password. The machine developed problems and the help desk guys re-imaged it upon request.

naturally the next day we got a ticket saying lastnights rsync didn't happen. Whoops... forgot to backup the script and the machine isn't in the backup loop.. Not wanting to admit anything or call the partner to ask them what the password is again, I decided to dd the file system and search for it.

At about the same time I finished re-writing the script, my grepper script found the old one and the password file! Get to avoid asking for a password and being 'that guy'. :)


The short here is if you delete something from XFS/EXT, or reformat the drive, there still is hope. I found the password file and the password days later on a live disk, using included GNU tools on the system (grep and dd).


Crez^D<96><90>^F^@^C^Drepo^D<96><90>^F^@^A^Coot^D<96><90>^F^@^A^Dsync^D<96><90>^F^@^E^Hexcludes^D<96><90>^F^@^E^Dpass^D<96><90>^F^@^@^Dsdir^D<96><90>^F^@^A^Ah^D<96><90>^F^@^A^Dtats^D<96><90>^F^@^B^Dudio^D<96><90>^F^@^@^Dtest^D<96><90>^F^@^A^Foolbox^D<96><90>^F^@^@^Duser^D<96><90>^F^@^@^@^F^B^@^@^@^@1^@MAR&@,^]^M^C<94>^@^A^@>^@^@>^C<94>^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<86>Q<86><90>^VG^@^@^@^@^@^@^@^@^@?^C^@^@<8b>W^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^C^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^
@^@^@^@^@^@application/x-shellscript^A<86>#! /bin/sh
--
CMD="rsync -avz --links --progress --stats --delete --force --chown=greg:greg --exclude-from /root/.rsyncexcludes "
USER=rezrepo
PASSFILE=/root/.rsyncpass
AUTH="--password-file /root/.rsyncpass"


$CMD $AUTH $SDIR rsync://$USER@$DIP/$DEST


^A<86>^A<88>^V^A<86>^@^@^@l^@^@^@^@^@^@^@^



here is the command I used:

grep -i -a -B10 'rsyncpass' /dev/mapper/centos7-root > findstuff.txt




-g



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



--


Greg Dickie
just a guy514-983-5400

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments  
   

Response from Greg Whynott @ May 3, 2017, 5:25 p.m.
XFS calls home? *gasp!*

On Wed, May 3, 2017 at 3:36 PM, Greg Dickie <greg@justaguy.ca> wrote:
Just ask the NSA.
Greg
On Wed, May 3, 2017 at 12:26 PM, greg whynott <greg.whynott@gmail.com> wrote:

I wrote a script some time back on a machine which was rsync'n to a partner using a password. The machine developed problems and the help desk guys re-imaged it upon request.

naturally the next day we got a ticket saying lastnights rsync didn't happen. Whoops... forgot to backup the script and the machine isn't in the backup loop.. Not wanting to admit anything or call the partner to ask them what the password is again, I decided to dd the file system and search for it.

At about the same time I finished re-writing the script, my grepper script found the old one and the password file! Get to avoid asking for a password and being 'that guy'. :)


The short here is if you delete something from XFS/EXT, or reformat the drive, there still is hope. I found the password file and the password days later on a live disk, using included GNU tools on the system (grep and dd).


Crez^D<96><90>^F^@^C^Drepo^D<96><90>^F^@^A^Coot^D<96><90>^F^@^A^Dsync^D<96><90>^F^@^E^Hexcludes^D<96><90>^F^@^E^Dpass^D<96><90>^F^@^@^Dsdir^D<96><90>^F^@^A^Ah^D<96><90>^F^@^A^Dtats^D<96><90>^F^@^B^Dudio^D<96><90>^F^@^@^Dtest^D<96><90>^F^@^A^Foolbox^D<96><90>^F^@^@^Duser^D<96><90>^F^@^@^@^F^B^@^@^@^@1^@MAR&@,^]^M^C<94>^@^A^@>^@^@>^C<94>^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<86>Q<86><90>^VG^@^@^@^@^@^@^@^@^@?^C^@^@<8b>W^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^C^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^
@^@^@^@^@^@application/x-shellscript^A<86>#! /bin/sh
--
CMD="rsync -avz --links --progress --stats --delete --force --chown=greg:greg --exclude-from /root/.rsyncexcludes "
USER=rezrepo
PASSFILE=/root/.rsyncpass
AUTH="--password-file /root/.rsyncpass"


$CMD $AUTH $SDIR rsync://$USER@$DIP/$DEST


^A<86>^A<88>^V^A<86>^@^@^@l^@^@^@^@^@^@^@^



here is the command I used:

grep -i -a -B10 'rsyncpass' /dev/mapper/centos7-root > findstuff.txt




-g



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



--


Greg Dickie
just a guy514-983-5400

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments  
   

Response from Greg Dickie @ May 3, 2017, 3:40 p.m.
Just ask the NSA.
Greg
On Wed, May 3, 2017 at 12:26 PM, greg whynott <greg.whynott@gmail.com> wrote:

I wrote a script some time back on a machine which was rsync'n to a partner using a password. The machine developed problems and the help desk guys re-imaged it upon request.

naturally the next day we got a ticket saying lastnights rsync didn't happen. Whoops... forgot to backup the script and the machine isn't in the backup loop.. Not wanting to admit anything or call the partner to ask them what the password is again, I decided to dd the file system and search for it.

At about the same time I finished re-writing the script, my grepper script found the old one and the password file! Get to avoid asking for a password and being 'that guy'. :)


The short here is if you delete something from XFS/EXT, or reformat the drive, there still is hope. I found the password file and the password days later on a live disk, using included GNU tools on the system (grep and dd).


Crez^D<96><90>^F^@^C^Drepo^D<96><90>^F^@^A^Coot^D<96><90>^F^@^A^Dsync^D<96><90>^F^@^E^Hexcludes^D<96><90>^F^@^E^Dpass^D<96><90>^F^@^@^Dsdir^D<96><90>^F^@^A^Ah^D<96><90>^F^@^A^Dtats^D<96><90>^F^@^B^Dudio^D<96><90>^F^@^@^Dtest^D<96><90>^F^@^A^Foolbox^D<96><90>^F^@^@^Duser^D<96><90>^F^@^@^@^F^B^@^@^@^@1^@MAR&@,^]^M^C<94>^@^A^@>^@^@>^C<94>^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<86>Q<86><90>^VG^@^@^@^@^@^@^@^@^@?^C^@^@<8b>W^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^C^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^
@^@^@^@^@^@application/x-shellscript^A<86>#! /bin/sh
--
CMD="rsync -avz --links --progress --stats --delete --force --chown=greg:greg --exclude-from /root/.rsyncexcludes "
USER=rezrepo
PASSFILE=/root/.rsyncpass
AUTH="--password-file /root/.rsyncpass"


$CMD $AUTH $SDIR rsync://$USER@$DIP/$DEST


^A<86>^A<88>^V^A<86>^@^@^@l^@^@^@^@^@^@^@^



here is the command I used:

grep -i -a -B10 'rsyncpass' /dev/mapper/centos7-root > findstuff.txt




-g



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe



--


Greg Dickie
just a guy514-983-5400

0 Plus One's     0 Comments  
   

Response from Jeff Klug @ May 3, 2017, 3:25 p.m.
Cool. Now nobody will ever know!


On Wed, May 3, 2017 at 12:26 PM, greg whynott <greg.whynott@gmail.com> wrote:

I wrote a script some time back on a machine which was rsync'n to a partner using a password. The machine developed problems and the help desk guys re-imaged it upon request.

naturally the next day we got a ticket saying lastnights rsync didn't happen. Whoops... forgot to backup the script and the machine isn't in the backup loop.. Not wanting to admit anything or call the partner to ask them what the password is again, I decided to dd the file system and search for it.

At about the same time I finished re-writing the script, my grepper script found the old one and the password file! Get to avoid asking for a password and being 'that guy'. :)


The short here is if you delete something from XFS/EXT, or reformat the drive, there still is hope. I found the password file and the password days later on a live disk, using included GNU tools on the system (grep and dd).


Crez^D<96><90>^F^@^C^Drepo^D<96><90>^F^@^A^Coot^D<96><90>^F^@^A^Dsync^D<96><90>^F^@^E^Hexcludes^D<96><90>^F^@^E^Dpass^D<96><90>^F^@^@^Dsdir^D<96><90>^F^@^A^Ah^D<96><90>^F^@^A^Dtats^D<96><90>^F^@^B^Dudio^D<96><90>^F^@^@^Dtest^D<96><90>^F^@^A^Foolbox^D<96><90>^F^@^@^Duser^D<96><90>^F^@^@^@^F^B^@^@^@^@1^@MAR&@,^]^M^C<94>^@^A^@>^@^@>^C<94>^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<86>Q<86><90>^VG^@^@^@^@^@^@^@^@^@?^C^@^@<8b>W^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^C^C^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^
@^@^@^@^@^@application/x-shellscript^A<86>#! /bin/sh
--
CMD="rsync -avz --links --progress --stats --delete --force --chown=greg:greg --exclude-from /root/.rsyncexcludes "
USER=rezrepo
PASSFILE=/root/.rsyncpass
AUTH="--password-file /root/.rsyncpass"


$CMD $AUTH $SDIR rsync://$USER@$DIP/$DEST


^A<86>^A<88>^V^A<86>^@^@^@l^@^@^@^@^@^@^@^



here is the command I used:

grep -i -a -B10 'rsyncpass' /dev/mapper/centos7-root > findstuff.txt




-g



To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe


0 Plus One's     0 Comments