Wednesday, October 14, 2020
Dban fact and fiction:
Dban is "good enough" for most but remember a few things. It does NOT guarantee a complete wipe. Wiping fully 2x is sufficient for total destruction with a few exceptions. Wiping 32 x is a waste of time and this was an idea that came about because older hard drives "MFM" and "RLL" would vary their writing depth based on the frequency of the drive. This could change for several reasons causing the write to go "deeper" in the media and thus more writes could be required to remove the data or not at all in most cases. Guttman is a myth beyond this. The need for this many writes is not true with modern drives, as the way data is stored, and the relation to the depth of the write is no longer dependant on the frequency of the drives write head. However loud noises can cause data to be mis-written due to the vibration of the sound and this is important but I digress. More to the point is that 32x (or whatever) is useless. On MBR disks typically a "quick format" only writes bytes 447 to 512 on the media which destroys the table of "file" pointers to the data and actually leaves the data in tact. When creating a partition the first 0 to 446 bytes *(see note) are written (or overwritten) and this creates/destroys the old data stored there but only there. Both instances of these operations destroy the database(s) stored there but leave bytes 513 to the end of the disk in tact and can be recovered later with a skilled hand. Note this is a little different with the GPT tables but basically the same effect. Only a Full Format will destroy the data and 2 full formats just to be safe. Format with Random data in case you missed anything (*caused by a loud noise hence the 2x formating) which would stand out from the rest of the Zero's you just wrote to the drive. This way any data missed due to _error_ looks like the rest of the noise you wrote randomly.
*Note: This is a "quick" overview of this topic for the purposes of debunking the "Dban" utility as the final solution to data destruction, and is not meant to be exact in the description of other related bytes written to the disk during the formatting of the drive. So don't shoot the messenger.
The exceptions are: Bad sectors "G" list, Bad Sectors "P" list, Wear Leveling:
G list:
Bad sectors can happen at any time from the life of the drive to the death and all drive have 2 tables. This is important because Dban is useless in this case. Dban does not have a function in which it performs any kind of checking to be aware of the bad sector table and does not perform any data recovery and wiping. The Bad sectors are managed by the drives firmware and completely hidden from Dban thus if you have a fragment of a image you shouldn't let's say, and you Dban the drive in which _error_ correction was performed at ANY time by the SMART service on the drive, this sector could have been remapped, marked as bad and left fragments of the image on the sector. Ie. One byte of a 512 byte sector is bad so smart "relocates" or copies this data (or as much as it can read) to a new good sector, but because it can't write, it marks this sector as bad, updates the Bad Sector table, and goes about the day and you never see a thing. Note the rest of the 511 bytes are still there. Dban will skip these sectors. This specific list of Bad Sectors is grown while in use and is specifically known as the "Grown List" or G list.
P list:
There is another set of bad sector tables known as the "Permanent List" and this is the list the drive is mapped with from the factory and can never be changed as it is hard coded to the firmware. This list is created at manufacturer time and because no drive is perfect in manufacturing thus mapping the default bad sectors is necessary to allow the drive to pass testing and be ready to function before it is released to the distributor as a working drive.
Wear Leveling:
On ALL digital media there are limitation as to how many times you can write to the same spot before it burns out. This is because apparently when storing data on an SSD of Flash media of any kind it involves pushing an electron through a "substrate" and lodging it there, which is read later as having or not having a charge (short version). This process leaves a residue behind. The residue will accumulate causing a buildup and eventually be unable to push an electron through and thus a "bad" sector will happen. To prevent this there is e technology called "Wear Leveling". Wear leveling is a small piece of software which will randomly move the disk writes
to to another location instead and update the File pointer table or File Allocation Table (FAT). The FAT is a database located at the first 447 to 512 bytes of the storage media which contains the name of the file and where the data associated with the filename is. Magnetic media does not use wear leveling because it can be written to indefinitely as long as there are no defects on the physical media and as explained before is uses the SMART software to update the "G" list if there is a problem. Wear leveling protects the longevity of writes to the drive in this fashion but never actually overwrites any data until all other available sectors are used and it must again (assuming the space is now empty) reuse the same spot. Rinse and repeat.
Conclusion:
To get around ALL of the problem associated with bad sector tables the only solution for total data destruction is not Dban, it is Full Disk Encryption deployed before you ever do ANYTHING else on the media. Full disk encryption is the ONLY way to secure your data from the life to the death of the device. Be careful that you select a sufficiently long password as you don't want a brute force attack to uncover your precious cat videos, and remember that if your disk is decrypted the record you have of "deleted" or "trashcan" files would then be recoverable. Wiping your "recycle bin" on an already Full Disk Encrypted device is a waste of time unless you are expecting to have you password breached, and don't want your deleted file recovered and that's just crazy talk. Don't be careless, use a good password when full disk encrypting and I recommend combining a security token such as RSA or a Yubikey that has no API for recovering the key once it has been written to the device with you current password to secure your data.
VPN Security, and Tor: Fact and Fiction.
I would say to be aware of a few things. First is that Tor is good enough if you follow a few rules you don't need to add a VPN:
VPN:
VPN's slow down an already slow Tor, and a VPN does not protect you at all from being tracked even if you paid all cash, despite what other may say. An VPN will create an encrypted tunnel from point A to B and prevent eavesdropping and is relatively fast as there are not layers of encryption happening at each server like Tor. This is great for logging onto a untrusted and maybe not very secure hotspot and tunneling your traffic to other regions on earth only to dump out your searches and browsing traffic at the location. Note that even if you paid all cash for the service it does not protect the IP source of the traffic at all. When looking at the data in wireshark you can see the source and destination IP no matter what. The VPN (if any hiding goes on) only wraps your data with their IP while in their data center but once that traffic re-emerges on the internet it is viewable by anyone with a copy of wireshark. It is true that your geographical location will change because the traffic is spilling out onto the internet from their VPN server hosted in the target location. In other words you don't need to pay cash for your VPN as it's only good to stop locals at the coffee shop from intercepting the data of where you are going, or to tunnel past your ISP and prevent DPI from identifying what type of traffic it is, and maybe censoring your access, or from conducting other types of molestation and therefore not worth going through all the trouble to pay cash. In no way does it hide the "Source" and "Destination" ip found in the Network Layer (3 OSI) from a protocol analyser like Wireshark. Once it leaves the VPN datacenter you are visible...period...or you would never get response from anyone on the internet obviously. Even finding a "bulletproof" VPN provider that does not perform logging is not enough on a Forensic level, you will still, be identified.
Tor:
Tor is VPN also but can have access to websites hosted within the Tor node network only other users of Tor have access to. If you were to leave the Tor network and say go to a non-tor hosted website you would again be visible at let's say google although your connection from Google would be encrypted. That means you conversation with Google would be visible only to google but your Source IP at Google would be Visible to Google employes, same problem as a regular VPN. Tor has servers a geographical location set up and are spaced apart and defined by a "boundary". If your traffic goes into a Tor server and does not cross this "Boundary" it will be traceable to the original source as it was never re-encrypted by another server (thus giving you that sweet, sweet privacy). That said you need to be aware that if you are in California, and browse to a site in Texas, your traffic may not have crossed the "Boundary" and was never re-encrypted thus leaving the source IP visible to other on that server or while in transit. You MUST cross a boundary when selecting a "Circuit" or you will be subject to immediate identification. Be sure to set up Tor to select a "circuit" in which this will apply carefully. Most recent versions of Tor are attempting to correct this and you may not have to set this up manually anymore but be aware of it with older versions as they are ALL subject to this flaw. You may only expect to remain invisible on the web if you stay within the Tor routing network, have forced a boundary to be crossed while selecting a circuit and have no other programs or plugins attached to the browser or accessing the internet on other ports. If you had Lastpass running, for example, and at some point it requested an update from the server on another port, this may expose your IP outside the Tor Circuit as Tor VPN's typically do not "wrap" the entire network stack in Linux or Windows and only covers the common Web access ports 80 and 443. This means that if Lastpass updates on port "1337" you will be immediately identified. Stay inside the Tor network, cross a geographical server boundary, and do not use plugins. Despite all of this, even if you are dutifully vigilant in your practice, but you are a regular at "foo.com", there is still screen and OS fingerprinting that can be used to follow you around inside the Tor routing system. Unlikely, but true. Stay on the move and don't fall into using TAILS (or whatever) at your "favorite" spot, and keep the default screen size, stay inside Tor exclusively, etc.
Remember that site "B" no matter who it is, will be the endpoint that CAN see you even if they aren't telling on you because they are hosting the content... You've been warned educated.
P.s. Let me add one more final thought: Be aware that if a server in or out of the Tor network has been ceased (Silk Road), or the servers Master Certificate Key was otherwise obtained (hackers) that this key can be used to decrypt all previous conversations with that server, with the exception of Perfect Forward Secrecy. PFS is an encryption standard in which each session with the server is re-keyed with a new key (short version). So if anyone were to get the key, they could only see that one session and NOT everything in the past from that IP. Look for PFS in the Sites Certificate details when accessing a site you want privacy from, and be aware that a "Gag" order may prevent you from knowing you are currently under surveillance...
Thursday, February 21, 2019
Take the most excellent quiz!
Take this coolness quiz I created as a google forms test!
Click here to take the quiz!
Good luck!
Thursday, November 2, 2017
Here is the Bitcoin Mining Captcha!
Click the "Verify Me" box and you can watch the Captcha solve a hash. This is not an Infected Link So therefore don't worry about clicking, Hackers aren't going to steal your money! 1M hashes equals a 0.00015279 XMR Payout which can be exchanged for Bitcoins! Click the "What is this?" link for more info on what's happening! Don't click the 'Submit' button because that is only for form submission, and this is just an example:)
Note: (difficulty: 106.96G, block reward: 3.14 XMR, payout: 70%, updated: February 21, 2019 - 12:04)
Note: (difficulty: 106.96G, block reward: 3.14 XMR, payout: 70%, updated: February 21, 2019 - 12:04)
Wednesday, June 14, 2017
Remove all of those annoying numbers and symbols from your music files!
Copy and paste the following into your Bash Terminal:
find /home/USER/Music -type f -iname '*mp3' -execdir rename 's/^\.\/[\d\s\-\_]+/\.\//' '{}' ';'
If your like me, I used to get my music from anywhere I could. This means there was no standard for "file naming conventions" and they always has weird and mostly irritating numbers and sometimes even symbols at the beginning of files! Well here is a Bash script one liner to get rid of all of them all at once!
Remember to make a backup of your primary music library, execute this command, and enjoy your newly renamed files! No need to worry about sub-folders. Just run the command as is, and it will look in all folders under your primary music folder. If you need help with other types of symbols that this didn't fix, just comment below and I will respond:)
**NOTE: Remember to change '/home/USER/Music' to match the location of your target files! You can use it to bulk rename files too: find /home/USER/Music -type f -iname '*mp3' -execdir rename 's/INSERT-PART-OF-FILENAME-TO-REPLACE-HERE/INSERT-FILENAME-TEXT-HERE/' '{}' ';'
Tuesday, February 2, 2016
Tired of having to sort through all of those .mp3 folders? Copy them all into one directory using this command!For Linux/Bash Users only!
Open a terminal and enter this:
find /home/USER/Music -name '*.mp4' -exec cp --backup=numbered -v -t /home/USER/NEWFOLDER/ {} +
This command will need to have the USER field replaced with your username, and also have the NEWFOLDER field replaced with the desired folder name.
This command will find all of the .mp3 files in the users Music folder and all it's subfolders, then execute a copy command which will place them all into one output folder called "NEWFOLDER" and give all duplicate files a name enumerated by a number in the new filename to distinguish them from one another.
For example, if the are three files with the same name, two of the will get numbers attached to the filename in numerical order. Ie. Music.mp3, 1Music.mp3, 2Music.mp3, etc..
**NOTE: Remember to change '/home/USER/Music' to match the location of your target files!
Setting up a Vmware Workstation with Spinrite in Linux:This is Experimental and only useful to determine if there is an error on the disk which needs to be fixed**
This is a question everyone seems to want to know so here goes the “light” version. This instruction assumes that you already have Vmware Workstation 12 and a legal copy of Spinrite in .iso format. Create a folder named '/home/USER/vmware/spinrite' and copy the Spinrite.iso into it. The purpose here is to use a Virtual machine to execute Spinrite without the need of another entire computer designated to the purpose of Data Recovery. This will also free up your production machine for other tasks. It's now time for you to know which device on your Linux machine that represents your USB drive when it is plugged in. Plug in your USB drive and open a separate command terminal. Enter: 'sudo fdisk -l' at the command line to list the currently mounted drives on the system and look for your USB disk. Mine shows as '/dev/sdb'. Remember this, and do not mistake your primary HDD! Make sure this is the device id of your usb drive! If your USB devices mount the same '/dev'sdb' every time then the rest is simple so here we go:
1. Run Vmware Workstation from the command line as root: 'sudo vmware'.
2. Create a new virtual machine and select “Typical” then click next.
3. Select “Use ISO image” and select the Spinrite.iso file to use a the source to install the Operating System. Note that there will never actually be an install of Spinrite so place the .iso inside the “vmware” folder as mentioned where it will stay permanently. Click Next.
4. Select the Guest Operating System as “Other” then select MSDOS. Click Next.
5. Now Choose the Virtual Machine Name as “Spinrite VM” with the default location of : “/home/USER/vmware/Spinrite”. Click Next.
6. Store the OS as a single file and the size will be .005 GB. This will create a 5MB disk. That's all we need! We are going to delete it anyway because this is just to create the VM. Click Next.
7. Uncheck the “Power on after Creation” checkbox, and click “Finish”.
8. Now edit the Virtual machine settings. Click on the Hard Disk to select it then click the “Remove” button at the bottom of the screen. We don't need a internal HDD – unless you are cloning but I digress from this for now. Edit the HDD and click on Advanced to turn off "Write Caching". Also remove the sound card, network adapter, floppy disk drive, and don't forget to turn off the “3d Support” under Display, and give this VM 1GB of RAM!
9. Now click the “Add” button and select “Hard Disk”. Then choose “SATA” and click next.
10. Select “Use a Physical Disk” and click next.
11. Remember your device id? Mine was '/dev/sdb' so I will use that and ever this into the “Device” field, and select “Use Entire Disk”. If you see a warning that you need to run as root you must STOP HERE and start Vmware Workstation as root, then start again at Step 1! Click next.
12. Now you should be at the “Disk File” menu which asks you if you want to use the “Spinrite-0.vmdk” file to store the settings. You will accept the default here and click Finish.
13. Now when you Power ON this VM immediately press F2 and enter the BIOS of the VM, navigate to the Boot menu and move the CDROM to the first in the Boot Priority. You can also right click the Spinrite VM tab, select Power, then Power on to Firmware, and change the setting from there. Then press F10 to “Save and Exit”.
14. All done! Your machine should reboot at which point your drive should be accessible.*
Now whenever you want to run Spinrite on a disk you can run Vmware as root (or else you can't access the '/dev/sdb' device!). Do NOT start the Spinrite VM yet, plug in your USB HDD, wait for the HDD of the Virtual machine to detect the USB Device (It will show a HDD when the USB is plugged in, no HDD when it isn't plugged in) Now start the Spinrite VM and fix any disk you want without any extra hardware!
*Note that this may not work on some DRV or XBox partitions either because I just have not figured it out yet, or Spinrite has an issue which prevents this from working properly. *UPDATE Spinrite has an 1TB limit as of Spinrite version 6.0.
**I have learned the hard way that once you install a Hypervisor on a computer, NEVER install another Hypervisor on the same system at the same time. I tell you all this just in case this type of stuff happens to you, and the symptoms I have described above matches your symptoms and you have been unable to use Spinrite to repair disks shortly thereafter the install of a second Hypervisor. This diagnosis also comes to you after not being able to repair disks for two years without any reason as to why it ceased to function. Without any notice the second Hypervisor "hooked" the Kernel thus overwriting a hook from the previous Hypervisor leaving the system crippled without any way of knowing or diagnosing the cause. I had to put 2+2 together. However since I had determined that the new Hypervisor was superior, I stayed with that for a long time trying to fix the problem. I never did. I found out because I went back to the old Hypervisor (just to stay familiar) and Viola! I could again repair disks! It was something I should have seen before the problem became muddled by time, and other installs. I had no reason to think that I should test the software against a new VM, and could think of no reason as to why it wouldn't work like usual. I almost downloaded a new copy of Spinrite thinking maybe mine was faulty somehow, but I figured it out and I still have the prestigious download count of "1" and wish to keep it. The above mentioned issues are the effect of the bad/unrepairable Kernel API/Hook that was made faulty by installing a second Hypervisor before completely uninstalling the first, AND REBOOTING. Even then, uninstalling and rebooting is not a guarantee that the old Hypervisor Hooks aren't still there and waiting to become faulty the moment you decide to switch to a different company for Virtualization. You may be able to switch back to the first hypervisor to get things working again, but be prepared for an full reinstall of the OS to fix the problem.
Subscribe to:
Posts (Atom)