Home Forums Reviews Tutorials Articles Register Search Today's Posts Mark Forums Read User Name Remember Me? Password Linux - Newbie This Linux forum is for members that are new to Linux.

Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices



Page 1 of 58 1 2 3 11 51 > Last » Search this Thread 09-11-2005, 09:39 PM #1 AwesomeMachine LQ Guru

Registered: Jan 2005 Location: USA and Italy Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint Posts: 5,513

Rep: Learn The DD Command Revised

How To Encrypt an 8.0 GB SDHC MicroSD Card was on 06-19-2011.



First Time visitors please reply.



How To Encrypt an 8.0 GB SDHC MicroSD Card

Put the card into an USB adapter. Such devices are not perfect. One might have to push the MicroSD card into the reader as far as it will go, and others might have to pull it back a millimeter or two. If the kernel does not detect a partition on a new card, it's detecting the USB adapter only. Adjust the card slighty, and replace the adapter if necessary. Should show some new device(s): Code: ls /dev/sd* /dev/sdb /dev/sdb1 Code: dd if=/dev/urandom of=/dev/sdb bs=4k Code: apt-get install cryptsetup Code: parted Deep inside, she knows she cannot attain masculinity, but she can attain masculinity deep inside! Code: cryptsetup --verbose --verify-passphrase luksFormat /dev/sdb1 Code: cryptsetup luksOpen /dev/sdb1 vol_1 Code: mkfs.xfs -imaxpct=3 /dev/mapper/vol_1 Code: mkdir /AES_Drive && mount /dev/mapper/vol_1 /AES_Drive Code: umount /AES_Drive && cryptsetup luksClose /dev/mapper/vol_1



Linux DD

The basic command structure is as follows:

Code: dd if=<source> of=<target> bs=<byte size> ("USUALLY" some power of 2, and usually not less than 512 bytes (ie, 512, 1024, 2048, 4096, 8192, 16384, but can be any reasonable whole integer value.) skip= seek= conv=<conversion>



Warning!! If you reverse the source and target, you can wipe out a lot of data. This feature has inspired the nickname "dd" Data Destroyer. Warning!! Caution should be observed when using dd to duplicate encrypted partitions.



Examples: duplicate one hard disk partition to another hard disk partition: Sda2 and sdb2 are partitions. You want to duplicate sda2 to sdb2. Code: dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=notrunc,noerror Be careful with order of if and of. You can write a blank disk to a good disk if you get confused. If you duplicate a smaller partition to a larger one, using dd, the larger one will now be formatted the same as the smaller one. And there will be no space left on the drive. The way around this is to use Code: rsync



To make an iso image of a CD: This duplicates sector for sector. MyCD.iso will be a hard disk image file of the CD. Code: dd if=/dev/hdc of=/home/sam/myCD.iso bs=2048 conv=sync,notrunc

Code: mkdir /mnt/myCD mount -o loop /home/sam/myCD.iso /mnt/myCD Code: cd /mnt/myCD

Code: dd if=/dev/fd0 of=/home/sam/floppy.image

Code: #!/bin/bash for n in `seq 7`; do dd if=/dev/urandom of=/dev/sda bs=8b conv=notrunc; done anti_scqid. Code: chmod +x anti_swqid



Best Laptop Backup: Purchase a laptop drive and an USB 2.0 drive enclosure (Total cost $100.00USD). Assemble the lappy drive into the external enclosure. Plug the external drive into the lappy USB port, and boot with The Knoppix live CD. Launch a terminal. This command will backup the existing drive: Code: dd if=/dev/hda of=/dev/sda bs=64k conv=notrunc,noerror Code: dd if=/dev/sda of=/dev/hda bs=64k conv=notrunc,noerror



This series will make a DVD backup of hard drive partition: Code: dd if=/dev/hda3 of=/home/sam/backup_set_1.img bs=1M count=4430 dd if=/dev/hda3 skip=4430 of=/home/sam/backup_set_2.img bs=1M count=4430 dd if=/dev/hda3 skip=8860 of=/home/sam/backup_set_3.img bs=1M count=4430 Code: wodim -dev=/dev/hdc --driveropts=burnfree /home/sam/backup_set_1.img Code: dd if=/media/dvd/backup_set_1.img of=/dev/hda3 bs=1M conv=sync,noerror Code: dd if=/media/dvd/backup_set_2.img of=/dev/hda3 seek=4430 bs=1M conv=sync,noerror

Code: dd if=/media/dvd/backup_set_3.img of=/dev/hda3 seek=8860 bs=1M conv=sync,noerror



If you wrote chat messages and emails to another girl, on your girlfriend's computer, you can't be sure the files you deleted are unrecoverable. But you can make sure if anyone were to recover them, that you wouldn't get busted. Code: dd if=/dev/sda | sed 's/Wendy/Janet/g' | dd of=/dev/sda



This command will overwrite the drive with zeroes Code: dd if=/dev/zero of=/dev/sda bs=4k conv=notrunc I just want to make sure my drive is really zeroed out!!

Code: dd if=/dev/sda | hexdump -C | grep [^00]



The following method of ouputting statistics applies to any dd command invocation. This is an example dd command so you can try it. Code: /bin/dd if=/dev/zero of=/dev/null count=100MB Code: ps aux | awk '/bin\/dd/ && !/awk/ {print $2}' | xargs kill -s USR1 $1 Code: 33706002+0 records in 33706002+0 records out 17257473024 bytes (17 GB) copied, 34.791 s, 496 MB/s Code: 58596452+0 records in 58596452+0 records out 30001383424 bytes (30 GB) copied, 60.664 s, 495 MB/s

Code: 74473760+0 records in 74473760+0 records out 38130565120 bytes (38 GB) copied, 77.3053 s, 493 MB/s Code: 100000000+0 records in 100000000+0 records out 51200000000 bytes (51 GB) copied, 104.193 s, 491 MB/s Code: dd if=/home/sam/file.file | clamscan - Windows users will find help in the second post, way at the bottom



FYI: duplicating smaller partition or drive to larger partition or drive; or vice versa:

Code: rsync -avH --exclude=/other_mount_point/ /mount_point/* /other_mount_point/ Code: rsync -avH --exclude=/mnt/destination_drive/ /* /mnt/destination_drive/

Code: grub-install update-grub



Making a NTFS partition, is not easy without using Windows based tools. I was formatting an external drive for my brother, who uses MS Windows XP. I wasn't going to admit Linux couldn't make a NTFS partition.



Make an ext3 partition on the drive. Open a hex editor and make a file containing Code: 07 ext3 partition to NTFS: Code: dd if=/home/sam/file.bin of=/dev/sdb bs=1 seek=450 count=1 Code: 0x1c2 Code: 0x83 Code: 0x07



The four primary partition type byte offsets are:

Code: 0x1c2=450 0x1d2=466 0x1e2=482 0x1f2=498 seek= parameter is changed from 450 to a one of the other values, it will change partition (hd0,1), (hd0,2), or (hd0,3) to NTFS type, rather than partition (hd0,0).



To be revised at a later date:

To make a bootable flash drive: Download 50 MB Debian based distro here:

http://sourceforge.net/projects/insert/



Plug in the thumb drive into a USB port. Do:

Code: dmesg | tail

Code: dd if=/home/sam/insert.iso of=/dev/sdb ibs=4b obs=1b conv=notrunc,noerror

End to be revised



This command will duplicate the MBR and boot sector of a floppy disk to hard drive image:

Code: dd if=/dev/fd0 of=/home/sam/MBRboot.image bs=512 count=2

Code: dd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror Do not reverse the intended source and target. It happens once in a while, especially to the inexperienced user. Notrunc means 'do not truncate the output file'. Noerror means to keep going if there is an error. Dd normally terminates on any I/O error.



Duplicate MBR, but not partition table. This will duplicate the first 446 bytes of the hard drive to a file:

Code: dd if=/dev/sda of=/home/sam/MBR.image bs=446 count=1 if and of, on the dd command line, reverses the direction of the write.



To wipe a hard drive: (Boot from a live CD distro to do this.)

Code: dd if=/dev/zero of=/dev/sda conv=notrunc



To overwrite all the free disk space on a partition (deleted files you don't want recovered): Code: dd if=/dev/urandom of=/home/sam/bigfile.file Code: no room left on device Code: rm bigfile.file



Sometimes one wants to look inside a binary file, looking only for clues. The output of the command line: Code: less /home/sam/file.bin Code: dd if=/home/sam/file.bin | hexdump -C | less Code: dd if=/home/sam/file.file | strings -n 8 -t d | less Code: dd if=/home/sam/JPEG.jpg bs=1w count=2 | hexdump -C Code: dd if=JPEG.jpg | hexdump -C Code: dd if=/dev/sda3 | hexdump -C | "grep 'ff d8 ff e0' | 'ff d9'" Code: grep Code: 0xba0002f Code: 0xbaff02a 195 035 183 and 196 079 658. (196 079 658) - (195 035 183) = rough idea of proper bs= and count= parameters. To find the proper count= figure: (<decinal offset of footer bytes> - <decimal offset of header bytes>) / <block size> = <number of blocks in the deleted JPEG file>. (195 035 183  196 079 658) = (1 044 475) / (bs=4096) = (254.998). That's really close to 255. If we could land exactly at the header bytes using bs=4096, we could use count=255. But I'm going to use count=257, because random chance dictates the probability of landing dead on the header bytes, using 2^x block size is remote. So we start reading before the header bytes.



We need to use skip= parameter to skip to our start point: 195 035 183 / bs=4096 = 47 616.011. We always round down, so dd will start reading before the beginning of the file. In this case we round down to skip=47615. The following writes a file containing the JPEG with some unwanted bytes before and after. Code: dd if=/dev/sda3 skip=47615 of=/home/sam/work_file.bin count=257 bs=4096 Code: hexdump -C work_file.bin | "grep 'ff d8 ff e0' | 'ff d9'" dd if=work_file.bin skip=<offset_of_first_header_byte_in_decimal_format> count=<offset_of_last_footer_byte_in_decimal_format +1> - <offset_of_first_header_byte_in_decimal_format> bs=1c of=JPG.jpg foremost.



The principle of file carving negates the need for Linux undelete programs. So if your from a MS Windows world, don't google for linux undelete, but rather, foremost NEXT ...



I put two identical drives in every one of my machines. Before I do anything that most probably spells disaster, like an untested command line in a root shell, that contains Code: find / -regex ?*.???* -type f | xargs rm -f "$1" Code: dcfldd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror

Code: dd if=/dev/sdb of=/dev/sda bs=4096 conv=notrunc,noerror

Code: dd if=/dev/sda2 (root) of=/home/sam/root.img bs=4096 conv=notrunc,noerror

Code: dd if /home/sam/root.img of=/dev/sda2 (root) bs=4096 conv=notrunc,noerror



How to make a swap file, or another swapfile on a running system:

Code: dd if=/dev/zero of=/swapspace bs=4k count=250000 mkswap /swapspace swapon /swapspace



How to pick proper block size:



Code: dd if=/dev/zero bs=1024 count=1000000 of=/home/sam/1Gb.file dd if=/dev/zero bs=2048 count=500000 of=/home/sam/1Gb.file dd if=/dev/zero bs=4096 count=250000 of=/home/sam/1Gb.file dd if=/dev/zero bs=8192 count=125000 of=/home/sam/1Gb.file

Read:

Code: dd if=/home/sam/1Gb.file bs=64k | dd of=/dev/null

Code: dd if=/dev/zero bs=1024 count=1000000 of=/home/sam/1Gb.file

Play with 'bs=' and 'count=', always having them multiply out to the same toal size. You can calculate bytes/second like this: 1Gb/total seconds = Gb/s. You can get more realistic results using a 3Gb file.



Rejuvenate a hard drive

To cure input/output errors experienced when using dd. Over time the data on a drive, especially a drive that hasn't been used for a year or two, grows into larger magnetic flux points than were originally recorded. It becomes more difficult for the drive heads to decipher these magnetic flux points. This results in I/O errors. Sometimes sector 1 goes bad, resulting in a useless drive. Try:

Code: dd if=/dev/sda of=/dev/sda



Make a file of 100 random bytes:

Code: dd if=/dev/urandom of=/home/sam/myrandom bs=100 count=1



Randomize data over a file before deleting it:

Code: ls -l

In this case it is 3769 Code: ls -l afile -rw------- ... 3769 Nov 2 13:41 <filename> Code: dd if=/dev/urandom of=afile bs=3769 count=1 conv=notrunc



Warning!! Do not write a partition image file to the same partition.

Code: dd if=/dev/sdb2 of=/home/sam/partition.image bs=4096 conv=notrunc,noerror

Code: dd if=/dev/sdb2 ibs=4096 | gzip > partition.image.gz conv=noerror Code: dd if=partition.image.gz | gunzip | dd of=/dev/sdb2 Code: < filename >.bz2

Code: dd if=/home/sam/partition.image of=/dev/sdb2 bs=4096 conv=notrunc,noerror

Code: dd if=filename of=filename conv=ucase

The Linux kernel makes a number a ramdisks you can make into ramdrives. You have to populate the drive with zeroes like so:

Code: dd if=/dev/zero of=/dev/ram7 bs=1k count=16384

Code: mke2fs -m0 /dev/ram7 4096

Code: debian:/home/sam # hdparm -t /dev/ram7 /dev/ram7: Timing buffered disk reads: 16 MB in 0.02 seconds = 913.92 MB/sec

Code: mkdir /mnt/mem mount /dev/ram7 /mnt/mem



Duplicate ram memory to a file:

Code: dd if=/dev/mem of=/home/sam/mem.bin bs=1024 Code: /dev/mem



Dd will print to the terminal window if you omit the Code: of=/dev/output

Code: dd if=/home/sam/myfile



To search the system memory:

Code: dd if=/dev/mem | strings | grep 'some-string-of-words-in-the-file-you-forgot-to-save-before-the-power-failed'

Code: mkdir /mnt/mem mount -t ramfs /dev/mem /mnt/mem dd if=/dev/zero > /mnt/mem/bigfile.file



You can get arrested in 17 states for doing this next thing. Make an AES encrypted loop device:

Code: dd if=/dev/urandom of=/home/sam/aes-drv bs=16065b count=100 modprobe loop modprobe cryptoloop modprobe aes losetup -e aes /dev/loop1 ./aes-drv password: mkreiserfs /dev/loop1 mkdir /aes mount -o loop,encryption=aes,acl ./aes-drv /aes password: mv /home/sam/porno /aes

Code: umount /aes losetup -d /dev/loop1 rmmod aes rmmod cryptoloop rmmod loop



If you are curious about what might be on you disk drive, or what an MBR looks like, or maybe what is at the very end of your disk:

Code: dd if=/dev/sda count=1 | hexdump -C

To see the end of the disk you have to know the total number of sectors, and the MAS must be set equal to the MNA. The helix CD has a utility to set this correctly. In the dd command, your skip value will be one less than MNA of the disk. For a 120 GB Seagate SATA drives

Code: dd if=/dev/sda of=home/sam/myfile skip=234441646 bs=512

So this reads sector for sector, and writes the last sector to myfile. Even with LBA addressing, disks still secretly are read in sectors, cylinders, and heads.

There are 63 sectors per track, and 255 heads per cylinder. There is a total cylinder count. 512_bytes/sector*63_sectors/track*255heads=16065*512bytes/cylinder=8,225,280_bytes/cylinder. 63_sectors/track*255_heads=sectors/cylinder. With 234441647 total sectors, and 16065 sectors per cylinder, you get some trailing sectors which do not make up an entire cylinder: 14593.317584812_cylinders/drive. This leaves 5102 sectors which cannot be partitioned, because to be in a partition you have to be a whole cylinder. It's like having part of a person. That doesn't really count as a person. These become surplus sectors after the last partition. You can't ordinarily read past the last partition. But dd can. It's a good idea to check for anything writing to surplus sectors. For our Seagate 120 GB drive, 234,441,647_sectors/drive - 5102_surplus_sectors = 234,436,545 partitionable sectors.

Code: dd if=/dev/sda of=/home/sam/myfile skip=234436545

Code: dd if=/dev/urandom of=/dev/sda bs=512 seek=234436545



Block size:

One cylinder in LBA mode = 255_heads*63_sectors/track=16065_sectors=16065*512_bytes=8,225,280_bytes. The b means '* 512'. 32130b represents a two cylinder block size. Cylinder block size always works to cover every sector in a partition, because partitions are made of a whole number of cylinders. One cylinder is 8,225,280 bytes. If you want to check out some random area of the disk: Code: dd if=/dev/sda of=/home/sam/myfile bs=4096 skip=2000 count=1000

Code: dd if=/home/sam/myfile of=/dev/sda bs=4096 seek=2000 count=1000

On source machine:

Code: dd if=/dev/hda bs=16065b | netcat < targethost-IP > 1234

Code: netcat -l -p 1234 | dd of=/dev/hdc bs=16065b

Code: netcat -l -p 1234 | bzip2 > partition.img Code: netcat -l -p 1234 | gzip > partition.img

Alert!! Don't hit enter yet. Hit enter on the target machine. THEN hit enter on the source machine.



Netcat is a program, available by default, on most linux installations. It's a networking swiss army knife. In the preceding example, netcat and dd are piped to one another. One of the functions of the linux kernel is to make pipes. The pipe character looks like two little lines on top of one another, both vertical. Here is how this command behaves: This byte size is a cylinder. bs=16065b equals one cylinder on an LBA drive. The dd command is piped to netcat, which takes as its arguments the IP address of the target(like 192.168.0.1, or any IP address with an open port) and what port you want to use (1234).



You can also use ssh. Code: dd if=/dev/sdb2 | ssh sam@192.168.0.121 "sudo dd of=/home/sam/sdb2.img" CONTINUED ... SEE NEXT POST



Dd is like Symantec Norton Ghost, Acronis True Image, Symantec Drive Image. You can perform disk drive backup, restore, imaging, disk image, cloning, clone, drive cloning, transfer image, transfer data, clone to another drive or clone to another machine, move Windows XP to a new hard drive, clone Windows XP, clone Windows, transfer Windows, hard drive upgrade, duplicate a boot drive, duplicate a bootable drive, upgrade your operating system hard drive, Tired of reinstalling WinXP Windows XP?



Copyright 2008, 2010 by AwesomeMachine.

All Rights Reserved. This post contains comprehensive documentation with examples for one of the most useful Linux/UNIX/Windows commands: dd. Dd is a bit-stream duplicator. If you have questions, post them. The latest addition,was on 06-19-2011.Put the card into an USB adapter. Such devices are not perfect. One might have to push the MicroSD card into the reader as far as it will go, and others might have to pull it back a millimeter or two. If the kernel does not detect a partition on a new card, it's detecting the USB adapter only. Adjust the card slighty, and replace the adapter if necessary. Should show some new device(s):Write random data to the drive:/dev/sdb is only an example.Learn to use parted, or I quit! Partition the card:Encrypt the partition with a good passphrase, one that's easy to remember, but hard to guess. DO NOT use the standard example, because everyone knows it:Open the encrypted device:Create a filesystem:Mount:Umount:Just a footnote, Laptops that went to sleep with the encrypted volume open, may wake up with it open!The basic command structure is as follows:Source is the data being read. Target is where the data gets written.Examples: duplicate one hard disk partition to another hard disk partition: Sda2 and sdb2 are partitions. You want to duplicate sda2 to sdb2.If sdb2 doesn't exist, dd will start at the beginning of the disk, and create it.If you duplicate a smaller partition to a larger one, using dd, the larger one will now be formatted the same as the smaller one. And there will be no space left on the drive. The way around this is to use, as described below.To make an iso image of a CD: This duplicates sector for sector. MyCD.iso will be a hard disk image file of the CD.You can mount the image like this:This will make the CD root directory the working directory, and display the CD root directory.This will duplicate a floppy disk to hard drive image file:If you're concerned about spies taking the platters out of your hard drive, and scanning them using superconducting quantum-interference detectors, you can always add a "for" loop for US Government DoD approved secure hard disk erasure. Copy and paste the following two lines into a text editor.Save the file asDon't run the program until you want to wipe the drive.Best Laptop Backup: Purchase a laptop drive and an USB 2.0 drive enclosure (Total cost $100.00USD). Assemble the lappy drive into the external enclosure. Plug the external drive into the lappy USB port, and boot with The Knoppix live CD. Launch a terminal. This command will backup the existing drive:This command will restore from the USB drive to the existing drive:If the existing disk fails, you can boot from the external drive backup and have your system back instantaneously.This series will make a DVD backup of hard drive partition:And so on. This series will burn the images to DVD+/-R/RW:and so forth. To restore the from the backup, load the DVDs in order, and use commands like these:Load another DVDLoad another DVDand so forth.If you wrote chat messages and emails to another girl, on your girlfriend's computer, you can't be sure the files you deleted are unrecoverable. But you can make sure if anyone were to recover them, that you wouldn't get busted.Where every instance of Wendy is replaced by Janet, over every millimeter of disk. I picked names with the same number of characters, but you can pad a smaller name with blanks.This command will overwrite the drive with zeroes... will return output of every nonzero byte on the drive. Play around with it. Sometimes drives don't completely zero out on the first try.The following method of ouputting statistics applies to any dd command invocation. This is an example dd command so you can try it.When you want to know how far dd has gotten throwing 100MB of 512 byte blocks of zeroes into digital hell, open another terminal and do:In the terminal running the dd command you will find something like this:If you enter the command again, you see more statistics:Againand so on ... Until the command completesHow To Scan a dd Bitstream for Viruses and Malware:You want to duplicate the root directory tree to another drive, but the other drive is larger. If you use dd, you will get a file system that is smaller then the larger destination drive. To duplicate files, not the file system: Format and mount the destination drive. Rsync will duplicate the files as files:You need to run:from a the rescue menu of an installation CD/DVD for the target to become bootable. If target was previously bootable, it remains bootable.Making a NTFS partition, is not easy without using Windows based tools. I was formatting an external drive for my brother, who uses MS Windows XP. I wasn't going to admit Linux couldn't make a NTFS partition.Make anpartition on the drive. Open a hex editor and make a file containingSave the file as file.bin. Change thepartition toWill change the partition type byte at offsetfrom Linux type:, to NTFS type:Please use a drive without important data on it. And, If you use a text editor to make the binary 07 file, you will ruin the existing partition table, because ascii 07 is two hexadecimal bytes (0x3037).The four primary partitionoffsets are:If the ddparameter is changed from 450 to a one of the other values, it will change partition (hd0,1), (hd0,2), or (hd0,3) to NTFS type, rather than partition (hd0,0).To make a bootable flash drive: Download 50 MB Debian based distro here:Plug in the thumb drive into a USB port. Do:Look where the new drive is, sdb1, or something similar. Do:Set the BIOS to USB boot, and boot.This command will duplicate the MBR and boot sector of a floppy disk to hard drive image:To clone an entire hard disk. /dev/sda is the source. /dev/sdb is the target:It happens once in a while, especially to the inexperienced user. Notrunc means 'do not truncate the output file'. Noerror means to keep going if there is an error. Dd normally terminates on any I/O error.Duplicate MBR, but not partition table. This will duplicate the first 446 bytes of the hard drive to a file:If you haven't already guessed, reversing the objects ofand, on the dd command line, reverses the direction of the write.To wipe a hard drive: (Boot from a live CD distro to do this.)This is useful for making the drive like new. Most drives have 0x00h written to every byte, from the factory.To overwrite all the free disk space on a partition (deleted files you don't want recovered):When dd ouputsall the free space has been overwritten with random characters. Delete the big file withSometimes one wants to look inside a binary file, looking only for clues. The output of the command line:is cryptic, because it's binary. For human readable output:You may also use:Recover deleted JPEG files. Look at the header bytes of any JPEG.The last two bytes are the footer.Using the JPEG header and footer bytes, search the drive. Command returns the offsets of the beginning and end of each deleted JPEG.Ifreturned JPEG header bytes at offset:and footer bytes at offset:Convert the hex offsets to decimal offsets, using one of the many logic capable calculators for Linux. Decimal offsets corresponding to the beginning and end of the JPEG areand. (196 079 658) - (195 035 183) = rough idea of proper bs= and count= parameters. To find the properfigure: ( - ) / = . (195 035 183  196 079 658) = (1 044 475) / (bs=4096) = (254.998). That's really close to 255. If we could land exactly at the header bytes using bs=4096, we could use count=255. But I'm going to use count=257, because random chance dictates the probability of landing dead on the header bytes, using 2^x block size is remote. So we start reading before the header bytes.We need to useparameter to skip to our start point: 195 035 183 / bs=4096 = 47 616.011. We always round down, so dd will start reading before the beginning of the file. In this case we round down to skip=47615. The following writes a file containing the JPEG with some unwanted bytes before and after.This sequence yields the desired JPEG.That's the way to get your hands dirty deep in digital data. But this process it automated in the file carving program,The principle of file carving negates the need for Linux undelete programs. So if your from a MS Windows world, don't google for, but rather,NEXT ...I put two identical drives in every one of my machines. Before I do anything that most probably spells disaster, like an untested command line in a root shell, that contains, I do:and duplicate my present working /dev/sda drive system to the /dev/sdb drive. If I wreck the installation on sda, I boot from a live CD distro, and do:And I get everything back exactly the same it was before whatever daring maneuver I was trying didn't work. You can really, really learn Linux this way, because you can't wreck what you have an exact duplicate of. You also might consider making the root partition separate from /home, and make /home big enough to hold the root partition, plus more. Then, To make a backup of root:To write the image of root back to the root partition, if you messed up and can't launch the X server, or edited /etc/fstab, and can't figure out what you did wrong. It only takes a few minutes to restore a 15 GB root partition from an image file:How to make a swap file, or another swapfile on a running system:This can solve out of memory issues due to memory leaks on servers that cannot easily be rebooted.How to pick proper block size:This method can also be used as a drive benchmark, to find strengths and weaknesses in hard drives:Read:Write:When dd finishes it outputs (total size)/(total time). You get the idea.Play with 'bs=' and 'count=', always having them multiply out to the same toal size. You can calculate bytes/second like this: 1Gb/total seconds = Gb/s. You can get more realistic results using a 3Gb file.Rejuvenate a hard driveTo cure input/output errors experienced when using dd. Over time the data on a drive, especially a drive that hasn't been used for a year or two, grows into larger magnetic flux points than were originally recorded. It becomes more difficult for the drive heads to decipher these magnetic flux points. This results in I/O errors. Sometimes sector 1 goes bad, resulting in a useless drive. Try:to rejuvenate the drive. Rewrites all the data on the drive in nice tight magnetic patterns that can then be read properly. The procedure is safe and economical.Make a file of 100 random bytes:/dev/random produces only as many random bits as the entropy pool contains. This yields quality randomness for cryptographic keys. If more random bytes are required, the process stops until the entropy pool is refilled (waggling your mouse helps). /dev/urandom does not have this restriction. If the user demands more bits than are currently in the entropy pool, it produces them using a pseudo random number generator. Here, /dev/urandom is the Linux random byte device. Myrandom is a file.Randomize data over a file before deleting it:to find filesize.In this case it is 3769duplicate a disk partition to a file on a different partition.This will make a file that is an exact duplicate of the sdb2 partition. You can substitue hdb, sda, hda, etc ... ORMakes a gzipped archive of the entire partition. To restore use:For bzip2 (slower,smaller), substitute bzip2 and bunzip2, and name the file.Restore a disk partition from an image file.Convert a file to uppercase:Make a ramdrive:The Linux kernel makes a number a ramdisks you can make into ramdrives. You have to populate the drive with zeroes like so:Populates a 16 MB ramdisk.puts a file system on the ramdisk, turning it into a ramdrive. Watch this puppy smoke.You only need to do the timing once, because it's cool. Make the drive again, because hdparm is a little hard on ramdrives. You can mount the ramdrive with:Now you can use the drive like a hard drive. This is particularly superb for working on large documents or programming. You can duplicate the large file or programming project to the ramdrive, which on my machine is at least 27 times as fast as /dev/sda, and every time you save the huge document, or need to do a compile, it's like your machine is running on nitromethane. The only drawback is data security. The ramdrive is volatile. If you lose power, or lock up, the data on the ramdrive is lost. Use a reliable machine during clear skies if you use a ramdrive.Duplicate ram memory to a file:The deviceis your system memory. You can actually duplicate any block or character device to a file using dd. Memory capture on a fast system, with bs=1024 takes about 60 seconds, a 120 GB HDD about an hour, a CD to hard drive about 10 minutes, a floppy to a hard drive about 2 minutes. With dd, your floppy drive images will not change. If you have a bootable DOS diskette, and you save it to your HDD as an image file, when you restore that image to another floppy it will be bootable.Dd will print to the terminal window if you omit thepart.will print the file myfile to the terminal window.To search the system memory:If you need to cover your tracks quickly, put the following commands in a script to overwrite system ram with zeroes. Don't try this for fun.This will overwrite all unprotected memory structures with zeroes, and freeze the machine so you have to reboot (Caution, this also prevents committment of the file system journal, and could trash the file system).You can get arrested in 17 states for doing this next thing. Make an AES encrypted loop device:to get the porno on the aes drive image.to make 'aes-drv' look like a 400 MB file of random bytes. Every time the lo interface is configured using losetup, according to the above, and the file 'aes-drv' is mounted, as above, the porno stash will be accessible in /aes/porno. You don't need to repeat the dd command, OR, the format with reiserfs, OR, the mv command. You only do those steps once. If you forget the password, there is no way to recover it besides guessing. Once the password is set, it can't be changed. To change the password, make a new file with the desired password, and move everything from the old file to the new file. Acl is a good mount option, because it allows use of acls. Otherwise your stuck with u,g,o and rwx.If you are curious about what might be on you disk drive, or what an MBR looks like, or maybe what is at the very end of your disk:Will show you sector 1, or the MBR. The bootstrap code and partition table are in the MBR.To see the end of the disk you have to know the total number of sectors, and the MAS must be set equal to the MNA. The helix CD has a utility to set this correctly. In the dd command, your skip value will be one less than MNA of the disk. For a 120 GB Seagate SATA drivesSo this reads sector for sector, and writes the last sector to myfile. Even with LBA addressing, disks still secretly are read in sectors, cylinders, and heads.There are 63 sectors per track, and 255 heads per cylinder. There is a total cylinder count. 512_bytes/sector*63_sectors/track*255heads=16065*512bytes/cylinder=8,225,280_bytes/cylinder. 63_sectors/track*255_heads=sectors/cylinder. With 234441647 total sectors, and 16065 sectors per cylinder, you get some trailing sectors which do not make up an entire cylinder: 14593.317584812_cylinders/drive. This leaves 5102 sectors which cannot be partitioned, because to be in a partition you have to be a whole cylinder. It's like having part of a person. That doesn't really count as a person. These become surplus sectors after the last partition. You can't ordinarily read past the last partition. But dd can. It's a good idea to check for anything writing to surplus sectors. For our Seagate 120 GB drive, 234,441,647_sectors/drive - 5102_surplus_sectors = 234,436,545 partitionable sectors.writes the last 5102 sectors to myfile. Launch midnight commander (mc) to view the file. If there is something in there, you do not need it for anything. In this case you would write over it with random characters:Will overwrite the 5102 surplus sectors on our 120 GB Seagate drive.Block size:One cylinder in LBA mode = 255_heads*63_sectors/track=16065_sectors=16065*512_bytes=8,225,280_bytes. The b means '* 512'. 32130b represents a two cylinder block size. Cylinder block size always works to cover every sector in a partition, because partitions are made of a whole number of cylinders. One cylinder is 8,225,280 bytes. If you want to check out some random area of the disk:Will give you 8,000 sectors in myfile, after the first 16,000 sectors. You can open that file with a hex editor, edit some of it, and write the edited part back to disk:Image a partition to another machine:On source machine:On target machine:Variations on target machine:makes a compressed image file using bzip2 compression.makes a compressed image file using gzip compression. I back up a 100 GB lappy disk on a desktop drive, over a lan connection, and the 100 GB compresses to about 4.0 GB. Most of the drive is empty, so it's mostly zeroes. Repetitive zeroes compress well.Netcat is a program, available by default, on most linux installations. It's a networking swiss army knife. In the preceding example, netcat and dd are piped to one another. One of the functions of the linux kernel is to make pipes. The pipe character looks like two little lines on top of one another, both vertical. Here is how this command behaves: This byte size is a cylinder.equals one cylinder on an LBA drive. The dd command is piped to netcat, which takes as its arguments the IP address of the target(like 192.168.0.1, or any IP address with an open port) and what port you want to use (1234).You can also use ssh.Dd is like Symantec Norton Ghost, Acronis True Image, Symantec Drive Image. You can perform disk drive backup, restore, imaging, disk image, cloning, clone, drive cloning, transfer image, transfer data, clone to another drive or clone to another machine, move Windows XP to a new hard drive, clone Windows XP, clone Windows, transfer Windows, hard drive upgrade, duplicate a boot drive, duplicate a bootable drive, upgrade your operating system hard drive, Tired of reinstalling WinXP Windows XP?Copyright 2008, 2010 by AwesomeMachine.All Rights Reserved. Last edited by AwesomeMachine; 06-19-2011 at 03:07 PM . Reason: revision Click here to see the post LQ members have rated as the most helpful post in this thread. 09-11-2005, 09:40 PM #2 AwesomeMachine LQ Guru

Registered: Jan 2005 Location: USA and Italy Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint Posts: 5,513

Original Poster

Rep: Code: ddrescue -b 2048 -d -e 20 -r 300 -v /dev/hdc /home/sam/dvd.iso log.txt /home/sam/dvd.iso won't mount, carve it: Code: foremost -t all -k 256 -v -b 2048 -i ~/dvd.iso -o ~/dvd/ /home/sam/dvd.



MS Windows Section



Use dd for drive cloning, backup, drive upgrading, and restore tasks. Boot a Windows XP machine with a Knoppix Linux live CD. Download Knoppix, burn the iso image file to a CD, boot with it, and clone drives. Drives are described to the dd command using device files. Boot into Knoppix, and open a root shell. It's in the penguin menu. (80 conductor grey ribbon cable) = /dev/hda for master; /dev/hdb for slave. SATA are /dev/sda and /dev/sdb.



The partitions on the first drive:

Code: fdisk -l /dev/hda Code: man fdisk Code: man parted



Many games written for Windows are virtually impossible back-up. This leaves only the original media. If it gets damaged, the user SOL. But there is a way to make back-ups using Linux. Download Adrienne Knoppix, and choose the option to copy it to a flash drive. Boot the flash-drive. If there are two optical drives available, load the CD/DVD source disk in one drive, and the blank disk in the other. Open a terminal from the menu, and get the root account by typing Code: su Code: ls /dev/hd* Code: ls /dev/sr* Code: wodim dev=/dev/hdc fs=16m speed=8 -raw96r driveropts=noburnfree,hidecdr -dummy -overburn -copy /dev/hdc/



Feel free to read the entire posts.

Dd is not presently able to clone Microsoft Windows Vista OEM Partitions.



How to make a bootable CD from a bootable floppy

Put the floppy in the floppy drive, and don't mount it, YET!



Code: dd if=/dev/fd0 of=/home/sam/floppy.img Code: mkdir /mnt/floppy.img mount -o loop /home/sam/floppy.img /mnt/floppy.img Code: cd /mnt/floppy.img/ rm <files you don't need> cp /home/sam/bios.bin Code: df -h Code: cd .. umount /mnt/floppy.img Code: mkisofs -o /home/sam/floppy.img.iso -b /home/sam/floppy.img /home/sam/floppy.img Code: wodim dev=/dev/hdc -sao driveropts=burnfree -dummy /home/sam/floppy.img.iso



You want to find out if your girlfriend is cheating on you, having cyber whoopie, or your a control freak. Even if the computer is secured with a password, you can boot with the: Knoppix Live CD and search the entire drive partition for text strings: Code: dd if=/dev/sda2 bs=16065 | strings -n 10 -t d | grep -i -B 20 -A 20 'luv U' dd command, for the text string specified between the single quotes in the grep command. Searching an entire disk partition several times can be time consuming. You might gift her a day at the spa. Females love that. That would keep her occupied while you invade her privacy. I highly recommend googling: "How can I tell if he's married?". This will produce a list of hits for sites females use to check whether or not their dates are married. You must click the link, go to the site, get the name of the organization running the site, google "<name of organization>", until you find the correct mailing address. Send a letter by certified mail instructing the organization to hereafter refrain from using a certain name for profit, because it doesn't belong to them, and they aren't entitled to profit by using it in a query or search result. State the specific name without a middle initial, and send a copy to any judge, with a cover letter, of course, and the enclosure marked COPY.



This particular command string prints the search results, preceded by the decimal offset of its location on the drive, to the screen. If the offfset is: 34,409,872 , we want manageable numbers, custom designed for speed and ease of use. The decimal disk offset is roughly 34 million, so the data we want to view is 34 MB into the partition. We divide 34,409,872 by some power of 2. Experience says 2^13 is about what we want, to get a quotient in the thousands. 34,409,872 /8192=~4200. The data we want is 8,192 4,200 byte blocks, OR, 4,200 8,192 byte blocks, into the partition. We check: 4200*8192=34406400; 34,409,872 -34406400=3472. This means the following command line will start reading 3,472 bytes before the string location.

Code: dd if=/dev/sda2 bs=4200 skip=8192 count=2 | strings > file.txt



[color="red"]This next thing does not work anymore, because the Linux kernel wised up a bit. I'm working on an alternate method.[color] You can search system memory with this method, by substituting Code: /dev/mem Code: /dev/sda2

Write system memory to a CD. This is useful for documenting memory contents without contaminating the HDD. I recommend using a CD-RW so you can practice a little. This doesn't involve dd, but it's cool. Code: wodim /dev=/dev/scd0 -raw driveropts=burnfree /dev/mem

Code: wodim --devices Code: dd if=/dev/hdd | hexdump -C | less Code: /dev/hdd Code: /dev/sda2

Code: dd if=/dev/hdd | strings -n 6 -t d | grep 'string'

list:

Code: '[[:alnum:]]' any alphanumeric characters '[[:alpha:]]' any alpha character '[[:digit:]]' any numeric character '[[:blank:]]' tabs and spaces '[[:lower:]]' any lower case alpha characters '[[:upper:]]' any uppercase alpha character '[[:cntrl:]]' ASCII characters 000 thru 037, and 177 octal '[[:graph:]]' [:alnum:] and [:punct:] '[[:punct:]]' any punctuation character ` ! ' # $ % ' ( ) * + - . / : ; < = > ? @ [ \ ] ^ _ { | } ~ '[[:space:]]' tab, newline, vertical tab, form feed, carriage return, and space '[[:xdigit:]]' any hex digit ranges('[a-d]' = any, or all abcd, '[0-9]' = any, or all 0123456789) Code: dd if=/dev/sda | hexdump -C | grep '[:punct:]' | less Code: hexdump -C

Back up your MBR:

Code: dd if=/dev/sda of=mbr.bin count=1

Code: dd if=boot.img of=/dev/fd0

Code: dd if=/dev/fd0 of=/home/sam/floppies/backup.bin conv=notrunc

Code: dd if=/home/sam/floppies/backup.bin of=/dev/fd0 conv=notrunc

Code: dd if=/dev/mem bs=1k skip=768 count=256 2>/dev/null | strings -n 8 dd will not duplicate or erase an HPA, OR, host protected area. Dd will erase a disk completely, but not as well as using the hardware secure erase, security erase unit command

Dd need not be black boxed like other inexpensive forensic software:

http://www.cftt.nist.gov/

For a low cost bootable CD based professional ghosting solution, that supports all operating systems and file systems:

http://www.feyrer.de/g4u/

Variation of dd for data rescue off defective media:

http://www.garloff.de/kurt/linux/ddrescue/

Department of Defense implementation of dd:

http://dcfldd.sourceforge.net/

Sdd is useful when input block size is different than output block size, and will succeed in some instances where dd fails:

http://linux.maruhn.com/sec/sdd.html

This is one of the best links I haven't written about dd:

http://www.softpanorama.org/Tools/dd.shtml



Copyright 2008 by AwesomeMachine.

All Rights Reserved.





Public Domain Copyright Material Begins Here:



Note that sending a SIGUSR1 signal to a running 'dd' process makes it

print to standard error the number of records read and written so far,

then to resume copying.



Code: $ dd if=/dev/zero of=/dev/null& pid=$! $ kill -USR1 $pid; sleep 1; kill $pid 10899206+0 records in 10899206+0 records out

So,

Code: dd if=/dev/sda of=/dev/sdb bs=1GB

bs=4b would give dd a block size of 4 disk sectors. 1 sector=512 bytes.

bs=4k would indicate dd use a 4 kilobyte block size. I have found bs=4k to be the fastest for copying disk drives on a modern machine.



OPERANDS The following operands are supported:

Code: if=file

Code: of=file

Standard output is the default.

seek=blocks Skip this many blocks in the output file.

Code: ibs=n

Code: obs=n

If no conversion other than Code: sync, noerror, and, notrunc

Code: cbs=n cbs= is omitted or given a value of 0, using Code: block or unblock

Code: ascii and asciib

Code: ebcdic, ebcdicb, ibm, and ibmb Code: files=n

Code: skip=n Code: iseek=n

Code: oseek=n Code: seek=n

Code: count=n

Code: conv=value

Code: conv=notrunc

Code: ascii

Code: asciib Code: ebcdic Code: dd conv=unblock

Code: ebcdicb

Code: dd conv=unblock

Code: ibm Code: dd conv=unblock

Code: ibmb Code: dd conv=unblock Code: ascii (or asciib), ebcdic (or ebcdicb), and ibm (or ibmb)

Code: lcase Code: ucase lcase and ucase symbols are mutually exclusive.

Code: swab

Code: noerror Code: sync Code: of=file Code: sync



ENVIRONMENT VARIABLES



The following environment variables affect the messages and errors messages of dd:



Code: LANG



Code: LANG



Code: LC_ALL



Code: LC_CTYPE



Code: LC_MESSAGES



Code: NLSPATH



Public Domain Copyright Material Ends Here How to rescue a CD or DVD, HDD, flash drive or photos:Ifwon't mount, carve it:Your files will be in directory:Use dd for drive cloning, backup, drive upgrading, and restore tasks. Boot a Windows XP machine with a Knoppix Linux live CD. Download Knoppix, burn the iso image file to a CD, boot with it, and clone drives. Drives are described to the dd command using device files. Boot into Knoppix, and open a root shell. It's in the penguin menu. (80 conductor grey ribbon cable) = /dev/hda for master; /dev/hdb for slave. SATA are /dev/sda and /dev/sdb.The partitions on the first drive:The manual page for fdisk:Use parted to make partitions:If one has trouble, leave a reply. Knoppix is slow, because it runs on a CD drive (1/1000 the speed of a HDD).Many games written for Windows are virtually impossible back-up. This leaves only the original media. If it gets damaged, the user SOL. But there is a way to make back-ups using Linux. Download Adrienne Knoppix, and choose the option to copy it to a flash drive. Boot the flash-drive. If there are two optical drives available, load the CD/DVD source disk in one drive, and the blank disk in the other. Open a terminal from the menu, and get the root account by typingNow typeIf /dev/hdb, /dev/hdc and possibly /dev/hdd come up, these are your optical drives, unless you have (2) ide hard drives. In that instance /dev/hdb would be a hard drive. If nothing comes up, typeThe possibilities are /dev/sr0 and sr1. If nothing comes up, your optical drive are /dev/scd0 and scd1. After the disks are loaded, typeIf there are no errors, hit the up arrow, remove '-dummy' and run the command.Feel free to read the entire posts.Put the floppy in the floppy drive, and don't mount it, YET!Mount /home/sam/floppy.img:Customize the image:or any other files you want on the CD. But don't exceed the 1.44 MB size of a floppy. Check space left in the mounted imageUnmount the floppy imageMake the .iso CD image file:Burn the iso file to a CD:This is a dummy burn, with the drive laser off. After you check the dummy run for errors, by looking at the program output, hit the up arrow, delete '-dummy', Enter. If you need a DOS boot floppy image file: [URL]http://www.freedos.org/[URL]You want to find out if your girlfriend is cheating on you, having cyber whoopie, or your a control freak. Even if the computer is secured with a password, you can boot with the:and search the entire drive partition for text strings:will search the drive partition specified in thecommand, for the text string specified between the single quotes in thecommand. Searching an entire disk partition several times can be time consuming. You might gift her a day at the spa. Females love that. That would keep her occupied while you invade her privacy. I highly recommend googling: "How can I tell if he's married?". This will produce a list of hits for sites females use to check whether or not their dates are married. You must click the link, go to the site, get the name of the organization running the site, google " ", until you find the correct mailing address. Send a letter by certified mail instructing the organization to hereafter refrain from using a certain name for profit, because it doesn't belong to them, and they aren't entitled to profit by using it in a query or search result. State the specific name without a middle initial, and send a copy to any judge, with a cover letter, of course, and the enclosure markedThis particular command string prints the search results, preceded by the decimal offset of its location on the drive, to the screen. If the offfset is:, we want manageable numbers, custom designed for speed and ease of use. The decimal disk offset is roughly 34 million, so the data we want to view is 34 MB into the partition. We divideby some power of 2. Experience says 2^13 is about what we want, to get a quotient in the thousands./8192=~4200. The data we want is 8,192 4,200 byte blocks, OR, 4,200 8,192 byte blocks, into the partition. We check: 4200*8192=34406400;-34406400=3472. This means the following command line will start reading 3,472 bytes before the string location.... and finish reading approximately 4,200 bytes after the string. This will net you 3.4k of disk contents before the search string, and 4.2k after. That's a 7.6k chunk of disk, plenty for what we're doing. With this method you search all the deleted files, any chat activity, Internet temporary files, and emails. It works regardless of what security has been deployed on the machine. It works with NTFS, ext2, ext3, reiserfs, swap, UFS, iso9660, and FAT partitions. But, it is illegal to use this method on a computer you aren't authorized to search. People can be sued, or imprisoned for performing unauthorized searches.[color="red"]This next thing does not work anymore, because the Linux kernel wised up a bit. I'm working on an alternate method.[color] You can search system memory with this method, by substitutingforWrite system memory to a CD. This is useful for documenting memory contents without contaminating the HDD. I recommend using a CD-RW so you can practice a little. This doesn't involve dd, but it's cool.to find the cdwriter:This method records raw, so you have to do a:to view the recorded memory. You can also employ the string search method above, substitutingforstring is any ascii sequence, hex sequence (must be separated with a space: '55 aa 09' searches for the hex string '55aa09'),list:... will return every line from theoutput that contains any punctuation characters specified above. It will not gather only punctuation characters.Back up your MBR:Put this on a floppy you make with:I back up floppies to a HDD. Floppies don't last forever, so I do:If my floppy fails, I can make unlimited copies:Here is a command line to read your BIOS, and interfaces:Copyright 2008 by AwesomeMachine.All Rights Reserved.Public Domain Copyright Material Begins Here:Note that sending a SIGUSR1 signal to a running 'dd' process makes itprint to standard error the number of records read and written so far,then to resume copying.BLOCKS and BYTES may be followed by the following multiplicative suffixes: c 1, w 2, b 512, kB 1000, K 1024, MB 1000*1000, M 1024*1024, GB 1000*1000*1000, G 1024*1024*1024So,Will use one gigabyte block sizes.bs=4b would give dd a block size of 4 disk sectors. 1 sector=512 bytes.bs=4k would indicate dd use a 4 kilobyte block size. I have found bs=4k to be the fastest for copying disk drives on a modern machine.OPERANDS The following operands are supported:Specifies the input path. Standard input is the default.Specifies the output path.Standard output is the default.Skip this many blocks in the output file.Specifies the input block size in n bytes (default is 512).Specifies the output block size in n bytes (default is 512).If no conversion other thanis specified, each input block is copied to the output as a single block without aggregating short blocks.Specifies the conversion block size for block and unblock in bytes by n (default is 0). Ifis omitted or given a value of 0, usingproduces unspecified results. This option is used only if ASCII or EBCDIC conversion is specified.operands, the input is handled as described for the unblock operand except that characters are converted to ASCII before the trailing SPACE characters are deleted.operands, the input is handled as described for the block operand except that the characters are converted to EBCDIC or IBM EBCDIC after the trailing SPACE characters are added.Copies and concatenates n input files before terminating (makes sense only where input is a magnetic tape or similar device).Skips n input blocks (using the specified input block size) before starting to copy. On seekable files, the implementation reads the blocks or seeks past them. On non-seekable files, the blocks are read and the data is discarded.Seeks n blocks from beginning of input file before copying (appropriate for disk files, where skip can be incredibly slow).Seeks n blocks from beginning of output file before copying.Skips n blocks (using the specified output block size) from beginning of output file before copying. On non-seekable files, existing blocks are read and space from the current end-of-file to the specified offset, if any, is filled with null bytes. On seekable files, the implementation seeks to the specified offset or reads the blocks as described for non-seekable files.Copies only n input blocks.[,value. . . ] Where values are comma-separated symbols from the following list:Do not truncate the output file.Converts EBCDIC to ASCII.Converts EBCDIC to ASCII using BSD-compatible character translations.Converts ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline withbeforehand.Converts ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline withbeforehand.Slightly different map of ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with ddbeforehand.Slightly different map of ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline withbeforehand. Thevalues are mutually exclusive. block Treats the input as a sequence of NEWLINE-terminated or EOF-terminated variable-length records independent of the input block boundaries. Each record is converted to a record with a fixed length specified by the conversion block size. Any NEWLINE character is removed from the input line. SPACE characters are appended to lines that are shorter than their conversion block size to fill the block. Lines that are longer than the conversion block size are truncated to the largest number of characters that will fit into that size. The number of truncated lines is reported. unblock Converts fixed-length records to variable length. Reads a number of bytes equal to the conversion block size (or the number of bytes remaining in the input, if less than the conversion block size), delete all trailing SPACE characters, and append a NEWLINE character. The block and unblock values are mutually exclusive.Maps upper-case characters specified by the LC_CTYPE keyword tolower to the corresponding lower-case character. Characters for which no mapping is specified are not modified by this conversion.Maps lower-case characters specified by the LC_CTYPE keyword toupper to the corresponding upper-case character. Characters for which no mapping is specified are not modified by this conversion. Thesymbols are mutually exclusive.Swaps every pair of input bytes. If the current input record is an odd number of bytes, the last byte in the input record is ignored.Does not stop processing on an input error. When an input error occurs, a diagnostic message is written on standard error, followed by the current input and output block counts in the same format as used at completion. If theconversion is specified, the missing input is replaced with null bytes and processed normally. Otherwise, the input block will be omitted from the output. notrunc Does not truncate the output file. Preserves blocks in the output file not explicitly written by this invocation of dd. (See also the precedingoperand.)Pads every input block to the size of the ibs= buffer, appending null bytes. (If either block or unblock is also specified, appends SPACE characters, rather than null bytes.)ENVIRONMENT VARIABLESThe following environment variables affect the messages and errors messages of dd:Provide a default value for the internationalisation variables that are unset or null. Ifis unset or null, the corresponding value from the implementation-dependent default locale will be used. If any of the internationalisation variables contains an invalid setting, the utility will behave as if none of the variables had been defined.If set to a non-empty string value, override the values of all the other internationalisation variables.Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single- as opposed to multi-byte characters in arguments and input files), the classification of characters as upper- or lower-case, and the mapping of characters from one case to the other.Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error and informative messages written to standard output.Determine the location of message catalogues for the processing of LC_MESSAGES.Public Domain Copyright Material Ends Here Last edited by AwesomeMachine; 06-19-2011 at 03:57 PM . 31 members found this post helpful. 09-11-2005, 09:52 PM #3 slackhack Senior Member

Registered: Jun 2004 Distribution: Arch, Debian, Slack Posts: 1,016

Rep: thanks, mate! thanks, mate! 09-11-2005, 10:15 PM #4 detpenguin Senior Member

Registered: Oct 2003 Location: lost in the midwest... Distribution: Slackware Posts: 1,098

Rep:

thanks man... personally...i found this thread *****AWESOME*****thanks man... 09-11-2005, 10:35 PM #5 Matir LQ Guru

Registered: Nov 2004 Location: San Jose, CA Distribution: Ubuntu Posts: 8,507

Rep: Great thread. New most of it, but I sure didn't a few years ago! Would've been great to have! 09-12-2005, 07:28 AM #6 sunsane LQ Newbie

Registered: Sep 2005 Location: INDIA Distribution: Mandrake Posts: 10

Rep: It's useful man!It's useful 09-12-2005, 08:36 AM #7 theYinYeti Senior Member

Registered: Jul 2004 Location: France Distribution: Arch Linux Posts: 1,897

Rep: Bookmarked! 09-12-2005, 08:47 AM #8 lord-fu Member

Registered: Apr 2005 Location: Ohio Distribution: Slackware && freeBSD Posts: 676

Rep: very cool............ 09-12-2005, 09:26 AM #9 logosys Member Contributing Member

Registered: Jul 2005 Location: Texas Distribution: RHEL4 - ES Posts: 178

Rep: can we stickify this? 09-12-2005, 10:28 AM #10 rickh Senior Member

Registered: May 2004 Location: Albuquerque, NM USA Distribution: Debian-Lenny/Sid 32/64 Desktop: Generic AMD64-EVGA 680i Laptop: Generic Intel SIS-AC97 Posts: 4,250

Rep: Straight to my personal HelpFiles folder. 09-12-2005, 10:35 AM #11 tuxdev Senior Member

Registered: Jul 2005 Distribution: Slackware Posts: 2,012

Rep: great article, however, explaining what "notrunc" is would be good. 09-13-2005, 03:53 AM #12 theYinYeti Senior Member

Registered: Jul 2004 Location: France Distribution: Arch Linux Posts: 1,897

Rep: True. Also, what is the difference between those:



dd if=/dev/urandom bs=1 count=100

dd if=/dev/urandom bs=100 count=1



Yves. 09-13-2005, 08:15 AM #13 addy86 Member

Registered: Nov 2004 Location: Germany Distribution: Debian Testing Posts: 332

Rep:

reads/writes 100 times 1 byte, whereas

dd if=/dev/urandom bs=100 count=1

reads/writes 1 time 100 bytes; the result is the same (but the latter is faster).

I've also experimented with blocksizes of below L1/L2 cache size but it wasn't faster, probably because the hard disk was the bottle neck, not the RAM dd if=/dev/urandom bs=1 count=100reads/writes 100 times 1 byte, whereasdd if=/dev/urandom bs=100 count=1reads/writes 1 time 100 bytes; the result is the same (but the latter is faster).I've also experimented with blocksizes of below L1/L2 cache size but it wasn't faster, probably because the hard disk was the bottle neck, not the RAM 09-14-2005, 02:59 PM #14 AwesomeMachine LQ Guru

Registered: Jan 2005 Location: USA and Italy Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint Posts: 5,513

Original Poster

Rep:



dd if=/dev/urandom bs=100 count=1



Is faster than



dd if=/dev/urandom bs=1 count=100



With urandom it is possible to read a byte size of 100. With /dev/random you need to do bs=1. Urandom does not have this restriction.



conv=notrunc is to prevent the output file from being truncated.



If you just want to write 1024 bytes to the beginning of other_existing__file.file, but leave the remainder of the file intact, this command will not work:

Code: dd if=/home/sam/existing_file.file bs=1k count=1 of=/home/sam/other_existing_file.file conv=notrunc tells dd to leave the remainder of the file in place. it is acknowledged by the author that:dd if=/dev/urandom bs=100 count=1Is faster thandd if=/dev/urandom bs=1 count=100With urandom it is possible to read a byte size of 100. With /dev/random you need to do bs=1. Urandom does not have this restriction.is to prevent the output file from being truncated.If you just want to write 1024 bytes to the beginning ofbut leave the remainder of the file intact, this command willwork:The output file will end up 1024 bytes long, and the rest will be gone. Addingtells dd to leave the remainder of the file in place. Last edited by AwesomeMachine; 08-22-2011 at 10:47 AM . 3 members found this post helpful. 09-15-2005, 05:45 AM #15 snakeo2 Member

Registered: Mar 2004 Location: Miami, Fl Distribution: Red Hat 9/fedora core3/ Posts: 273

Rep: great article, im in the process of expanding my / partition and this artilcle will help clear a lot of questions, thanks



Page 1 of 58 1 2 3 11 51 > Last »

Tags backup, best, clonezilla, cloning, command, data, dd, disk, drive, duplicate, erase, explanation, formatting, ghost, hard, image, iso, memory, ping, popular, recover, recovery, rescue, search, security, stick, upgrade, usb, wipe



Thread Tools Search this Thread Show Printable Version Email this Page Search this Thread:



Advanced Search

Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts On BB code is On Smilies are Off [IMG] code is HTML code is Off Forum Rules



Similar Threads Thread Thread Starter Forum Replies Last Post Learn The DD command AwesomeMachine Linux - Newbie 17 08-17-2006 04:22 AM The best way to learn? iz3r Programming 7 02-06-2005 11:00 PM Best way to learn Linux from the command line patpawlowski Linux - General 2 03-01-2004 03:37 PM I want to learn C. KptnKrill Programming 14 12-18-2003 01:03 PM Best way to learn.... InEeDhElPlInUx Linux - Newbie 5 10-11-2003 01:02 AM



All times are GMT -5. The time now is 01:25 AM .

