Sonntag, 6. November 2011

Winetricks

Trotz der großen Auswahl an Programmen, die heutige Linux-Distributionen von Haus aus mitbringen und über sogenannte “Repositories” anbieten, oder die sich für lau aus dem Netz besorgen lassen, kommt fast jeder so genannter “Normalbenutzer” mal in die Verlegenheit, ein Windows-Programm unter Linux starten zu wollen. In diesem Fall bleibt ihr oder ihm nur die Möglichkeiten, Windows in einer virtuellen Maschine zu installieren, oder das OpenSource-Programm Wine zu verwenden.

Wine steht dabei für “Wine Is Not an Emulator” und ermöglicht es, Programme, die für Windows bestimmt sind, unter Linux/Unix auszuführen. Leider funktioniert das längst nicht mit allen Programmen und manchmal klappt es nur mit zusätzlichen “Tricks”. Der Grund dafür kann daran liegen, dass immer mehr Windows-Programme zusätzliche Laufzeitkomponenten benötigen um selbst unter Windows zu arbeiten. Diese werden natürlich auch unter Linux mit Wine benötigt.

Winetricks ist eine Skript mit dem verschiedene Redistributable-Runtime-Bibliotheken, beispielsweise die MS Visual Basic und C++ Runtime Libraries, heruntergeladen und installiert werden können. Zusätzlich lassen sich mit diesem Skript auch einige Einstellungen, wie die Wahl der Windowsversion, anpassen. Winetricks wird nicht installiert, es genügt die Datei aus dem Internet herunterzuladen und an einem beliebigen Ort zu speichern. Das funktioniert über die Kommandozeile mit dem Befehl:

$ wget http://www.kegel.com/wine/winetricks

Gegebenenfalls wird noch cabextract benötigt, dass sich mit dem Befehl:

$ apt-get install cabextract

unter Debian-basierte Distributionen installieren lässt. Zum Ausführen von Winetracks genügt anschließend die Eingabe:

$ sh winetricks

Darauf erscheint ein Fenster, in dem man die gewünschten Bibliotheken und Einstellungen auswählt und mit OK bestätigt. Danach werden die ausgewählten Bibliotheken heruntergeladen und installiert. Das Auswählen und installieren kann auch durch z. B.:

$ sh winetricks corefonts vcrun6

direkt ausgeführt werden.

Ein Aktuelles Beispiel für Windows-Programme, die sich mit Winetricks zum Laufen bringen lassen, ist das Programm “AdWords Editor”, dass es nur für Windows gibt. Dieses Programm ist bis zur Version 7.0 relative problemlos unter Wine gelaufen. Mit der Version 7.5 ging das leider nicht mehr. Beim Starten kommt die Fehlermeldung “Microsoft Visual C++ Runtime Library – Runtime Error!”.

Erst mit der Installation der MS Visual C++ 2005 und MS Visual C++ 2005 SP1 Bibliotheken über

$ sh winetricks vcrun2005 vcrun2005sp1

startet das Programm wieder. Dabei ist auf die genaue Reihenfolge der Installationen zu achten!

Mittwoch, 26. Oktober 2011

Using the Knocked Daemon

Knockd is a daemon that listens to communication events at the link layer for sequences then acts on them.

what this means, is that you can have the machine running the daemon not even "listen" on the ports (a port scan will not see them as open) and still make it do something you need, from a single command to a full script.

Using this technique you could trigger the server to perform any sort of operation you need of it remotely (across the internet) without exposing your home network.

For more information, even though the installation instructions don't work anymore, feel free to have a look at my youtube series about using it on the DD-WRT router.


Overview

what we are going to do is:
  • setup the server to quietly "listen" for the safe sequences without revealing that its doing so (ports are closed).
  • send the sequences using the commonly used telnet command from anywhere other then our home network.
  • the server will perform some example operations:
    • Email us that a sequence was received and from what IP.
    • wake up a neighbor computer
    • connect to a Linux based router (Tomato) and open an additional port for 20 seconds and auto close it.


Prerequisites
  1. Some TCP/IP knowledge is assumed - to keep this page from becoming a two volume book on TCP, i assume you understand how TCP/IP works and what are IP addresses and ports.
  2. You will need to forward the ports used for the knocking on the router to the machine that is running the daemon.
    While less efficient then simply having it on the router, by no means less secure (remember they can be none-open ports and this will still work).
  3. You have already setup Email support for the server using the "Email setup for alerts" guide.

Installation

The installation could not be simpler on Debian, all you need to do is issue:
sudo aptitude install knockd

Configure sequences

Once you have installed the package, you need to configure the action you want the daemon to take and the triggering sequences for them.

Edit the configuration file /etc/knockd.conf:
sudo vim /etc/knockd.conf
This file has some examples of options you may wish to use.
The first default option i leave in there is:
[options] UseSyslog
This configuration will make knocked use the syslog facility instead of a file (like it used to be on the router).

Enabling the daemon & starting it

In order to make the daemon enabled for use & startup, it is necessary to edit its syscrl file:
sudo vim /etc/default/knockd
Iin the file, change the "start" parameter to 1 to enable it:
START_KNOCKD=1
start the daemon by issuing:
sudo /etc/init.d/knockd start

WOL sequence

WOL is an acronym for Wake On Lan.
This enables you to tell other computers on the network to turn on when you send a specially formed packet to them over the network.

Prerequisites
  • You need to enable WOL for the target computer - This is usually a combination of a BIOS setting and a driver setting (out of the scope of this article).
  • You have already setup Email support for the server using the "Email setup for alerts" guide.

What we will configure is:
  • The sequence that will trigger the script.
  • A script that will take the MAC of the computer we want to turn on as a parameter.
  • The script will Email us that the command has been issued to what MAC and from what IP.

Doing it this way, has the added advantage of informing you (via Email) that the command was executed (instead of blindly hoping that it did).
Also in case you didn't do the knocking, you both will at lease know that is happened and who did.

Required package

You need to install WOL support for your server, on Debian based systems this is accomplished by simply issuing:
sudo aptitude install etherwake

The sequence template

Edit the knockd configuration file:
sudo vim /etc/knockd.conf
Add this entry to it:
[WOL-Laptop]     sequence    = 04,04,04,2012,2012,2012,03,03,03     seq_timeout = 50     command     = /home/aviad/knockd_wake_up.sh my_laptop %IP%     tcpflags    = syn
This entry will:
  • Make a new sequence that will require 3 telnet commands on the ports 04, 2012 and 03.
  • Give the sequence 50 seconds to be completed from the moment it was started.
  • Call the script we will create later and pass to it two parameters:
    • The name of the computer we want to wake
    • The IP address of the computer that made the knocking request.

The wake up script

Create a new file:
sudo vim /home/aviad/knockd_wake_up.sh
Make this its content:
#!/bin/bash   email_admin_script() {     echo "To: youremail@some-domain.com" > $temp_email_file     echo "From: servername@some-domain.com" >> $temp_email_file     echo "Subject: Server-name knockd WakeUp" >> $temp_email_file     echo "" >> $temp_email_file     echo $e_body >> $temp_email_file     echo "" >> $temp_email_file     echo "As requested by:" >> $temp_email_file     echo $from_ip >> $temp_email_file     /usr/sbin/ssmtp -t < $temp_email_file     echo "Sent an Email to the Admin" }   temp_email_file=/tmp/wake_tmp.txt computer="$1"   case "$computer" in     my_lapotp)         e_body="Woke up computer Aviad's E6400 Laptop"         MAC=00:81:52:6B:3D:AE         ;;     my_desktop)         e_body="Woke up computer Aviad's E6400 Laptop"         MAC=00:18:65:3F:1C:86         ;; esac     from_ip="$2"   /usr/sbin/etherwake $MAC   email_admin_script rm $temp_email_file  
This script will take the two parameters that where passed by the knockd daemon, as its input and use them to:
  1. Wake a computer using the etherwake command.
  2. Send an email to the administrator to let him know that the command has been issued and by whom.

In order to add more computers, simply add additional sequences to the knockd configuration file and expand the "case" statement branches in the script.

start the daemon by issuing:
sudo /etc/init.d/knockd start
Restart the daemon for the settings to take affect, by issuing:
sudo /etc/init.d/knockd restart
That's it you should be done.

VPN sequence

This sequence will communicate with the home router to open additional ports for VPN communication in response to a knock.
By doing this the security of the VPN connection is increased without really changing how the VPN works (i.e. there is no need to change the client and generate certificates etc).

Note: I don't recommend novices to try this one as its way overly complex and is very tied to a specific setup, beyond the fact that it isn't very well tested...
It is possible to have the script change locally set iptable rules instead of the routers rules, but wheres the fun in that?
But in all seriousness I have invested to much energy into this page to simply throw it away...

Also I am truly sorry that it isn't well tested, but since I started writing this page to the time I finished, I have stopped using Tomato as the router firmware and the "home server" as the VPN server, and have returned to using the router as the VPN server with knockd on DD-WRT. This was made possible with the aid of the firmware development kit, which enabled me to inject the knockd and ssmtp packages to the mega firmware on my 8MB flash device (WRT54GSv3 thanks to Mike:)
With that said, once you have the SSMTP and the knockd packages installed on the router (if you managed to rape ipkg to work or by using the dev kit), you can use my personal configuration page here.

Preface

  1. I used a router with the Tomato firmware.
  2. I am assuming that you have telnet\SSH access to the router.
  3. The router is connected to the ISP by using an L2TP type of a dialer.

My configuration file looks like this:
(Of course i have changed the actual port sequence)
 [enable-VPN]  sequence = 02,02,02,01,01,01,2010,2010,2010  seq_timeout = 120                      start_command = /home/aviad/enable_vpn.sh %IP%                      cmd_timeout = 20                      stop_command = /home/aviad/disable_vpn.sh
This configuration will:
  • Set the window of opportunity, of the sequences' completion, to 120 seconds.
  • Listen to a sequence of *three knocks on ports 2, 1 and 2010 (this order is deliberate to throw ports scanners off track).
  • Once the sequence has been detected, run the script in /root/enable_vpn.sh and pass to it the IP address of the requester (thats the meaning of the %IP% parameter).
  • Wait for 20 seconds and issue the /root/disable_vpn.sh script.

*We are using "three knocks" because that's the way telnet (our call back home program) acts when it cant reach the port, it tries three times before quitting.

Create the action scripts

Our script will connect back to the router that's running the Tomato firmware and inject three additional iptable rules.
(it was possible to do on the local server, but this is way cooler ^_^)

The enable script

Ccreate a new script and edit it:
sudo vim /home/aviad/enable_vpn.sh
Make this its content:
#!/bin/bash ssh root@192.168.1.1 '/sbin/gpio disable 2; /usr/sbin/iptables -t nat -I PREROUTING 2 -d $(nvram get l2tp_get_ip) -s '$1' -p tcp --dport 1723 -j DNAT --to 192.168.1.51:1723; /bin/sleep 2; /sbin/gpio disable 3; /usr/sbin/iptables -I wanin 1 -p tcp -s '$1' -d 192.168.1.51 --dport 1723 -j ACCEPT;  /usr/sbin/iptables -t nat -I POSTROUTING 2 -s 192.168.1.0/24 -d 192.168.1.51 -p tcp --dport 1723 -j SNAT --to $(nvram get wan_ipaddr)'   exit 0

lets brake it down:
  • ssh root@192.168.1.1 - connects to the router using SSH (we will setup no password login in the next step) and run the commands in the quotations.
  • gpio disable 2; - turns the router front led on with the orange color (disable=turn on, enable=turn off)
  • iptables -t nat -I PREROUTING 2 -d $(nvram get l2tp_get_ip) -s '$1' -p tcp --dport 1723 -j DNAT --to 192.168.1.51:1723 - adds a rule to the NAT table that anything destined for the L2TP was IP with the ogf port 1723 will be directed to the 192.168.1.51 address on the same port.
  • sleep 2; gpio disable 3; - sleep for 2 seconds and activate the white led on the front of the router.
  • iptables -I wanin 1 -p tcp -s '$1' -d 192.168.1.51 --dport 1723 -j ACCEPT; - add a rule to allow communication from the "wan input" interface to the server only to the address that was passed from the knockd request. (the $1 script parameter is equal to the %IP% parameter)
  • iptables -t nat -I POSTROUTING 2 -s 192.168.1.0/24 -d 192.168.1.51 -p tcp --dport 1723 -j SNAT --to $(nvram get wan_ipaddr) - get the IP address of the "WAN" interface (witch apparently isn't the same as the dialer address and source NAT anything from the 192.168.1.0/24 network to the server.
  • exit 0 - inform the parent script that the script ran correctly.

The disable script

Create a new script and edit it:

sudo vim /home/aviad/disable_vpn.sh
Make this its content:
#!/bin/bash ssh root@192.168.1.1 '/sbin/gpio enable 2; /usr/sbin/iptables -t nat -D PREROUTING 2; /bin/sleep 2; /sbin/gpio enable 3; /usr/sbin/iptables -t nat -D POSTROUTING 2; /usr/sbin/iptables -D wanin 1'   exit 0
Lets brake it down:
  • ssh root@192.168.1.1 - connects to the router using SSH and run the commands in the quotations.
  • /sbin/gpio enable 2; - turns the router front led on with the orange color off.
  • /usr/sbin/iptables -t nat -D PREROUTING 2; - delete the rule that enabled NAT from out to in.
  • /bin/sleep 2; /sbin/gpio enable 3; - sleep for 2 seconds and turn off the white led on the front of the router.
  • /usr/sbin/iptables -t nat -D POSTROUTING 2; - delete the the rule that enabled PAT to the server.
  • /usr/sbin/iptables -D wanin 1 - delete the rule that enabled communication from the "wan input" to the server.
  • exit 0 - inform the parent script that the script ran correctly.

Make the scripts executable:
sudo chmod +x /home/aviad/enable_vpn.sh sudo chmod +x /home/aviad/disable_vpn.sh
Restart the daemon for the settings to take affect, by issuing:
sudo /etc/init.d/knockd restart

Setting up a password-less trust

Because we are using SSH to send the commands from the server to the router, and we want the operation to be performed automatically, we need to make the router trust our server.
We will do this by creating a public key on our server and inject it into the router.

  • Create the RSA public key for the root user (that's the user that will be executing the commands) :
    sudo ssh-keygen -t rsa
  • Get the content of the public key:
    sudo cat /root/.ssh/id_rsa.pub
  • The output should look something like:
    ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA8vBGqHoqY+VPl3VcxmmSprhc0YkhuhPKc1gmKtZ/sLmEB1v3TVeNuTpow0eBdG1JGVTiOfrlkTcrOK1Tm5Kw/I4mb1eAzLKl4Iix8W++FXYon4m2/ygINUhcioBARWe4FzajsS35FnKkjwy8ALGtKk+cU7yRojGr0zMzvVH1XwIzaLiQTKx0xOoftpr5a8Xja84JcMKI2K25mHTvs+XF8ELVqFqTOPNFohAgZBzSlZMuKuAiV4N7z+Htw47sGDo3Dos8ZwHGnOgPQQYTahKGP0dTkk8TBZ7yvQXEsB8X4EmSN2UoWcmqabUA2p7WCnD/s+lDWWEIJpxFmM8LY834mw== root@lenny64template 
(This isn't "my key" so don't even think about it...:)
  • Because the router will erase the key on every reboot, we need to create a startup script that will re inject the key back, once the router has been restarted.
  • Log into the routers' web-GUI, and go to:
    1. "Administration"
    2. "Scripts"
    3. "WAN Up"
      a. Select the "Word Warp" check box (just so the out put would be clearer to read)
      b. Type in: echo "past-your-server's-public-key-here"
      c. After the key, append to the end of the line, " > /root/.ssh/authorized_keys" (note the space), so that this command will add the servers public key into the authorization keys , as shown in the picture below.

That's it, assuming you have the
VPN server setup you should be able to connect to it once the sequence has been triggered.


Testing the configuration

Once you have a sequence configured you should be able to see the daemons reaction to knocks by viewing the syslog output while doing some knocks.

For this example we will assume you have the sequence of 02, 01 and 2010 set.
You can do that by issuing the tail command on the server:
sudo tail -f /var/log/syslog
On a remote network, open a command prompt and telnet back to the network:
telnet your.routers.ip.or.dns 02 telnet your.routers.ip.or.dns 01 telnet your.routers.ip.or.dns 2010
Remember, telnet tries 3 times automatically if the port is unreachable for communication.

The output of a successful knock should look like:
May 30 10:01:10 lenny64template knockd: 62.213.132.44: enable-VPN: Stage 1 May 30 10:01:13 lenny64template knockd: 62.213.132.44: enable-VPN: Stage 2 May 30 10:01:14 lenny64template knockd: 62.213.132.44: enable-VPN: Stage 3 May 30 10:01:18 lenny64template knockd: 62.213.132.44: enable-VPN: Stage 4 May 30 10:01:21 lenny64template knockd: 62.213.132.44: enable-VPN: Stage 5 May 30 10:01:25 lenny64template knockd: 62.213.132.44: enable-VPN: Stage 6 May 30 10:01:29 lenny64template knockd: 62.213.132.44: enable-VPN: Stage 7 May 30 10:01:32 lenny64template knockd: 62.213.132.44: enable-VPN: Stage 8 May 30 10:01:33 lenny64template knockd: 62.213.132.44: enable-VPN: Stage 9 May 30 10:01:33 lenny64template knockd: 62.213.132.44: enable-VPN: OPEN SESAME  

Donnerstag, 13. Oktober 2011

Unix ist einfach - zum Tode von Dennis Ritchie

Dennis Ritchie
Dennis Ritchie -
Wie erst jetzt bekannt wurde, ist der US-amerikanische Informatiker Dennis MacAllstair Ritchie am vergangenen Wochenende nach langer Krankheit in Murray Hill gestorben. Ritchie wurde 70 Jahre alt.

Über 40 Jahre beeinflusste er maßgeblich die Entwicklung der Informationstechnologie. Er gilt als einer der treibenden Kräfte hinter dem Aufstieg von Unix: Zusammen mit Ken Thompson schrieb und portierte er die erste Version von Unix. Für Unix konzipierte er mit Thompson und Brian Kernighan die Programmiersprache C. Das von ihm und Kernigham geschrieben Buch "The C Programming Language" gilt als eines der einflussreichsten Informatikwerke, das Heerscharen von Programmierern den Einstieg in ihren Beruf bereitete. Außerdem war Ritchie für die ersten sechs Ausgaben des "Unix Programmer's Manual" verantwortlich, ehe das Handbuch in "Unix Time-Sharing System" unbenannt wurde.

Dennis M. Ritchie wurde am 9. September 1941 in Bronxville geboren. Sein Vater arbeitete als Elektroingenieur bei den Bell Labs von AT&T, wo Ritchie 1967 im neuen "Computing Science Research Center" seine Karriere startete, nachdem er in Harvard Physik und angewandte Mathematik studiert hatte. Parallel zur Arbeit in den Laboratorien promovierte Dennis Ritchie in Harvard bei Patrick C- Fischer mit einer Arbeit über rekursive Programmierstrukturen, "Program Structure and Computational Complexity".

Noch während seiner Studienzeit hatte Ritchie erste Erfahrungen mit MULTICS (Multiplexed Information and Computing Service) gesammelt, einem Betriebssystem, das am Massachussetts Institute of Technology (MIT) für General Electric entwickelt worden war. Die Erfahrungen mit MULTICS veranlassten Ritchies Kollegen Ken Thompson, einen anderen Weg zu beschreiten, wie Ritchie erzählte: "We were a bit oppressed by the big system mentality. Ken wanted to do something simple. Presumably, as important as anything was the simple fact that our means were much smaller - we could get only small machines with none of the fancy Multics hardware. /.../ Multics wasn't there for us anymore, but we liked the feel of interactive computing that it offered."

Das Resultat dieser Arbeit führte im Jahre 1969 zur Entstehung von Unics, das für die DEC PDP-7 konzipiert wurde. Für dieses System entwickelte Dennis Ritchie auf der Basis von B eine Programmiersprache, die er zunächst "New B", dann C nannte. Während dieser Zeit entwickelte sich aus Unics das Unix Time-Sharing System First Edition (V1), das 1970 auf einer PDP-11 installiert wurde. Der große Wurf kam 1973, als Dennis Ritchie und Ken Thompson das Betriebssystem in der Sprache C komplett neu schrieben. Diese Version begründete die weitreichende Portabilität von Unix; allerdings dauerte es 10 Jahre, bis die erste Portierung auf ein DEC fremdes Rechnersystem erfolgte. Dennis Ritchie und Steve Johnson portierten Unix 1978 auf die Interdata 8/32.

Für diese Leistung erhielten Ritchie und Thompson im Jahr 1998 die National Medal of Technology. Unix und C hätten die Computertechnologie stimuliert und so die führende Rolle der USA im Informationszeitalter gefördert, erklärte der damalige US-Präsident Bill Clinton. Ritchies Erklärung wurde legendär und lautete vollständig: "Unix is simple and coherent, but it takes a genius – or at any rate a programmer – to understand and appreciate the simplicity."

Zeit seines Berufslebens bis zum Ausscheiden im Jahre 2007 arbeitete Dennis Ritchie bei den Bell Labs. Als größte Ehre hielt er einen Labs-Eintrag am Gebäude, in dem die Erfindung des Transistors und die Entwicklung von Unix als die beiden wichtigsten Geschenke der Labs an die Welt bezeichnet wurden. Zusammen mit Ken Thompson und Brian Kernigham erhielt Ritchie zahlreiche Preise und Ehrenwürden, darunter den Turing Award 1983 und den Japan-Preis. Das von ihm mitentwickelte Unix hielt Ritchie nicht für das ultimative Betriebssytem, sondern ermahnte seine Kollegen 1984 in der Zeitschrift Commications of the ACM:

"The greatest danger to good computer science research today may be excessive relevance. If we can keep alive enough openess to new ideas, enough freedom of communication, enough patience to allow the novel to prosper, it will remain possible for a future Ken Thompson to find a little-used Cray/1 computer and fashion a system as creative, and as influential, as Unix." (Detlef Borchers) / (jk)

Montag, 27. Juni 2011

VMware ESX vSphere resize disk

Every now and then I need to resize (usually extend/enlarge) a disk attached to a Virtual Machine. I have tried several methods to do this over the years (including combinations of VMware Converter, third party partition manager apps, diskpart etc) but none have been as efficient as the method I discovered during recent VMware training for my VCP4 exam.

One of the new features of vSphere is the ability to resize disks without having to shut down the Virtual Machine. This was previously impossible in VI3. This greatly speeds up the resizing process which can be executed in a couple of stages:
1) Use the vSphere Client to edit the settings of the Virtual Machine in question. Select the hard disk and modify it's provisioned size as appropriate. Click OK to apply these changes - resizing the .vmdk file.
2) Verify that the .vmdk has been resized by opening the Management Console -> Disk Management to find the unallocated space on the disk that resides in the .vmdk (distinguished by the black colour in the legend at the bottom.) In this case you can see I have increased the size by 5GB.

Right click on the disk (in this case 'Disk 0') and select properties. On the Volumes tab make a note of the unallocated space, in my case it is 5122MB.

If you are using Windows Server 2003 or earlier versions of Windows (Note: Windows Server 2008 now allows you to resize a disk through Disk Management in the Computer Management Console/Server Manager even if this disk is the Boot/Page File disk) download Dell's EXTPART and extract it on the server that contains the disk you want to resize. Navigate to c:\dell\ExtPart (the default extracted location) and run extpart.exe. When prompted enter the the Windows drive letter of the disk on the Virtual Machine e.g. c:. When prompted for the size to extend the partition by enter the number noted down earlier (I used 5122 in this example.) After doing so the disk should be resized. You can check this by opening the Management Console -> Disk Management and verifying the size of the partition.


NB - If you receive the following error:
"Unable to connect to c: or it does not exist"
There are a couple of workarounds that you could try.
1) Close the Management Console (if it is open) and try extpart.exe again.
2) Try restarting the VM in safe mode and then run extpart.exe. This is not ideal but it is still easier than other methods I have tried to resize .vmdk files.

Mittwoch, 22. Juni 2011

How to convert MBps to IOPS or calculate IOPS from MB/s

Most SSDs have two speed ratings for reading as well as for writing. The first rating is the sustained MB/s performance, which is the main marketing most manufacturers use. The second rating is the 4K Random IOPS performance, which gives a much better idea of how the drive will perform in the real world. The sustained MB/s rating is the sequential transfer rate the SSD will maintain continuously, such as over a period of 30 seconds. The 4K Random IOPS on the other hand is how many 4K (4096 byte) operations the drive will handle per second with each block being read or written to a random position.
Usually when a Random 4K IOPS figure is given, it may state that this is at a certain queue depth, such as 4, 16, 32 or 64. With a queue depth of 4, this means that there are 4 separate threads taking place with the drive, each thread independently running its own transfers. With the use of Native Command Queuing (NCQ), the SSD can handle these threads simultaneously to improve the overall throughput compared to running a single thread. While many hard disks use native command queuing to line up read/write operations to minimise seek times between each read/write operation, SSDs can read from and write to multiple NAND cells simultaneously, where as the read/write head in a hard disk can only be in one physical place at any time.
While it is nice to see how many IOPS a drive is capable of, it is also useful to see how this translates into actual throughput or even vice versa. For example, most benchmark tools such as CrystalDiskMark and AS SSD report the random 4K performance in throughput, i.e. MB/s, while the SSD’s specifications usually rates the 4K performance in IOPS.
To see how to translate MBps into IOPS and vice versa, we need to do a little math:
IOPS = (MBps Throughput / KB per IO) * 1024
Or
MBps = (IOPS * KB per IO) / 1024
So let’s say we have an SSD claiming a Random 4K write speed of 20,000 IOPS and it achieves 76.2MB/s in the CrystalDiskMark with the QD32 write test.
To convert the 76.2MB/s to IOPS, we perform the following calculation:
IOPS = (76.2 / 4) * 1024
IOPS = 19.1 * 1024
IOPS = 19,558.4
To see what throughput we need to achieve to match the actual 20,000 IOPS claim, we can perform this calculation in reverse:
MBps = (20,000 * 4) / 1024
MBps = 80,000 / 1024
MBps = 78.125MB/s
Note that each manufacturer uses its own method of coming up with their SSD IOPS ratings. Besides separate IOPS ratings for read and write speeds, a given SSD can behave quite different depending on the type of data being read or written, as well as the duration this transfer takes place. For example, an SSD that achieves 5,000 Random 4K IOPS sustained write over a period of 30 seconds may only achieve 1,000 IOPS sustained write over a period of 5 minutes. For SSDs using the SandForce processor, the compressibility of the data also has an impact, so two benchmark tools may show completely different IOPS readings if one tool sends uncompressible data and the other tool sends highly compressible data and also if one runs the benchmark for longer duration than the other.

Donnerstag, 9. Juni 2011

Unix shell unter Windows

C:\Documents and Settings\takoloko>dir | grep *.jpg

Mit den UnxUtils könnt ihr eine ganze Reihe nützlicher Linux Programme bzw. Befehle in der Windows Command Line verwenden. Beispiele: gawk, grep, less, unrar, wget.
Eine komplette Liste der UnxUtils Befehle findet ihr auf der Homepage.

Installation UnxUtils

  1. Downloaden – Die UnxUtils sind in einer zip-Datei gepackt, die ihr euch bei Sourceforge downloaden könnt. Die Datei heisst UnxUtils.zip (nicht UnxUtilsSrc.zip!).
  2. Entpacken – Als nächstes müsst ihr die Datei in C:\ entpacken.
  3. Pfad anpassen – Um die Befehle in der Command Line nutzen zu können, müsst ihr noch die Pfad Variable anpassen. Rechsklick auf den Arbeitsplatz -> Eigenschaften -> Reiter “Erweitert” -> unten auf “Umgebungsvariablen” -> im unteren Feld die Variable “Pfad” auswählen -> Doppelklick auf die Variable -> am Ende der Zeile ;C:\usr\local\wbin der Pfadvariable hinzufügen.
  4. Fertig

Links:

Autostart Windows 2008 Server [All Users]

%ProgramData%\Microsoft\Windows\Start Menu\Programs\Startup

Mittwoch, 8. Juni 2011

Add a new Partition on opensuse 11.4

1. Plan the new filesystem. Where would you like to use the new space? Do

df

to print a summary of free/used space on each of the existing mounted partitions. Do

du

on selected directories to find their size.

For example, I would consider using new hard drive space in one of the following mount points:

/usr/local
/home
/home/share/downloads
/usr/local/mp3s
/usr/local/dos_data

Here is Details.

/usr/local is suposed to survive any upgrade of Linux. It is nice to have it on a separate partition because I can even reformat other partitions without affecting my local contents stored in /usr/local. I surely want it of the type "ext2" or perhaps "reiserfs".

/home contains user data. Surely, it is the data that deserves the most care. It is obviously supposed to survive any upgrade of Linux. Wow, obviously I want it on a separate partition. The type is normally "ext2" or "raiserfs".

/usr/local/mp3s is a non-standard Linux directory. I may keep my MP3 (music) files there. Those tend to be large.

/usr/local/dos_data. Another non-standard directory. If I dual boot, I would consider making an extra partion of the type "DOS FAT32" or similar so as to share files between MS Windows and Linux transparently (both ways). I would configure all the Windows-based programs to use this "drive" as the default location for all user-generated files. I could even have "mp3s", "cds" and other such directories in this location. The serious drawback of this approach--MS Windows may insist on messing up with this partion on re-install.

2. Partition the new hard drive. For example, if my new harddrive is the slave on the second IDE interface (perhaps the "fourth" IDE drive), I could use:

cfdisk /dev/hdd

or the more old-fashioned (and standard) tool:

fdisk /dev/hdd

If your drive is not "hdd" adjust the above command as needed.

hda -- first ide master (whole disk)
hdb -- first ide slave
hdc -- second ide master
hdd -- second ide slave
sda -- first scsi (whole disk)
sdb -- second scsi (whole disk)
...
sdp -- sixteenth scsi (whole disk)

For other disks, consult /usr/src/Linux/Documentation/devices.txt.

Most of the time, Linux partitions to be of the type ext2 ("Linux").

Partitioning can be tricky--if you never have done it, read man fdisk and man cfdisk. It is very easy to delete a partition with all your data. Make sure you know which disk you are working with!

fdisk or cfdisk does not make any changes to the hard drive until I write the new partition layout. So if I make a bad mistake, I can always quit without writing. I write the layout to the drive only when I am completely done.

3. Format each partition. For example, to format the first partition, while checking for bad blocks (-c), I would do:

mkfs -c -t ext2 /dev/hdd1

4. Test the new partitions around. Mount the new partitions manually (the mount directory must exist and be empty). Copy a bunch of files to each partition. View/edit a couple of random files. Delete them all.

5. Copy data. Optional--only if you would like to move data from an old partition to a new partition. Go to the single-user mode (init 1). Mount the new partition manually. Copy the data from the old partition to the new partition. Careful with the old data, you probably don't want to lose it if you made a mistake, so I wouldn't delete it yet--I rename the top level directory appropriately. E.g.,

cp -R /usr/local/ /mnt/hdd1/
mv /usr/local/ /usr/local.old.backup_of_2005-04-21

6. Edit the file /etc/fstab. Modify it to reflect your new filesystem layout. Perhaps, insert the mountpoint for the new partition(s) or modify any old mountpoints as needed. For example, if moving /usr/local to its own partition, I would need to add to add a line like this:

/dev/hdd1 /usr/local ext2 defaults 1 2

7. Reboot and test. The alternative to reboot is to unmount old and mount new mount points. For example:

umount /usr/local
mount -a

but hard reboot may be a more rigorous test of the new layout.

8. Remove old data. After a few days, when I have the confidence everything is really working fine