gzip without gzip

I hit a few problems today when I was just trying to deflate this blog a little. Server bandwidth is a critical factor and 4 megabytes for every pageload is just too huge. The biggest part is tribute to many pretty pictures which I thought I had already downsized.

When it comes to php-based pages and text content though, gzip is number one on the list of promising candidates. Simple thing, install one of those generic gzip plugins from the plugin archive and you’re done – or so I thought. Turns out my server does have the zlib library, but it is not responding. After some experimenting with the plugin code, I realized that whichever way I tried to enable gzip buffered output would not work. Unfortunately, this seems to be quite a common problem.

The solution is found in the archives of php.net. In the community exchange for the ‘ob_gzhandler’-object, someone posted this smart workaround. If you break it down, it just buffers all output until the very end, then gets it, compresses it manually using zlib and returns it combined with the necessary markings. And even better – this one works even if zlib refuses to take on the more simple jobs!

Combined with a howto on wholepage filtering published on w-shadow.com, I ended up with a compact piece of code that can be inserted into an generic wordpress plugin:

function ws_set_up_buffer () {
    ob_start('ws_filter_page');
}
add_action('wp', 'ws_set_up_buffer', 10, 0);
 
function ws_filter_page ($html) {
    header("Content-Encoding: gzip");
    $gz_data = $html;
    $gz_size = strlen($html);
    return "x1fx8bx08x00x00x00x00x00".substr(gzcompress($html, 4), 0, -4).pack('V', crc32($html)).pack('V', $gz_size);
}

The first of the two functions is called before wp starts to gather the data to be displayed. All it does is redirect all output into a buffer with a special id, so we can identify it later on.

Next up is a callback function. It is executed after the blog framework has finished loading data and has flushed (emptied) the cache into the $html-variable. From there, all data is taken and shoved into the gzip compressor. The rest is just makeup so browsers will recognize the packet as genuine gzipped http.

What may be missing is a check, if the browser of the current visitor is capable of receiving gzip. This can be achieved using a conditional like this one in the callback function:

if(strstr($HTTP_SERVER_VARS['HTTP_ACCEPT_ENCODING'], 'gzip'))

The final code would look like this:

function ws_set_up_buffer () {
    ob_start('ws_filter_page');
}
add_action('wp', 'ws_set_up_buffer', 10, 0);
 
function ws_filter_page ($html) {
    if(strstr($HTTP_SERVER_VARS['HTTP_ACCEPT_ENCODING'], 'gzip')) {
        header("Content-Encoding: gzip");
        $gz_data = $html;
        $gz_size = strlen($html);
        return "x1fx8bx08x00x00x00x00x00".substr(gzcompress($gz_data, 7), 0, - 4).pack('V', crc32($gz_data)).pack('V', $gz_size);
    }
    else
    {
        return $html;
    }
}

I have just tested this implementation, works fine.

* Had problems with HTTP_ACCEPT_ENCODING not being properly readable, if encoding does not happen, just comment out the outer IF-construction!

A final word of warning: Things can become difficult if your wordpress installation includes other plugins that try to do wholepage filtering. See w-shadows blog post for details. You need to pay attention to the nesting of the plugins, as the gzip plugin must be the last one to tamper with the data. There will be no readable html left after it has finished.

Hey,you found it

I decided to move this blog over to a personal domain (you have already arrived here, so this part works) before I get too accustomed to the limited possibilities of wp.com, and because I have full control over the file host this way. This means I can finally include those EAGLE-files that wordpress would not let me upload.
While I am still figuring out the design and configuration, most content is already available here, so enjoy your stay!

Oh, and about the name…I wanted to make this a bit more personal, and there’s this neat little coincidence that my last name ist spoken like the greek letter µ in both english and german. What could be better, related to scientific matters ;-) I’m just not sure if the net and its creepy-crawly inhabitants (google and friends) like it, but we’ll see.

The old blog will still be available for a few weeks.

UV + vacuum = better pcbs?

Inspired by a project published in the magazine ELEKTOR about converting one of those little membrane air pumps used on fishtanks and stuff into a vacuum pump, I went and opened up the pump that came with my pcb etching machine. Exciting! For this one there isn’t even any modification necessary as the inlet is connected to the regulator by a hose which can be easily redirected. True, it might need another hole in the case to poke the hose through, but if that is all there’s to it…will try tonight!

That hose is acutally a bypass outlet. If the valve is open, pressure goes out of the tube instead of the port on the side of the pump. The REAL inlet is just below the black nozzle, it’s a small hole in the orange pump casing. Look at picture below!

The resulting vacuum can then be used to evacuate the air out of a clear plastic foil sandwich containing the pcbs with attached layouts, effectively removing all air bubbles and pressing the layouts against the photo resist – like it is done when curing epoxide resins. I am pretty curious if this will solve the problems I’ve been having with blurred pcb traces.

Regent R-002 pump innards

Inlet hole

Read more about the idea and its use over at the original authors (french) website. Just look for “Pompe A Vide” after the redirect.

LUKS meets badblocks

Most linux users know the famous badblocks tutorial because almost every harddrive starts to produce bad sectors at some point in its life. In some cases they correspond to real damages on the platter surfaces, but sometimes the drive logic has just made an error and blocked further reading of the sector. To give the drive a chance to reallocate the sector to unused space OR to rethink its decision (yes, some bad sectors just disappear after this) you will need to overwrite the sector, best with zeroes. Before this, it is always smart to probe the sector for its occupation, and which file it has been assigned to, if so. The tutorial describes a way to do this.

Things get more complicated if the drive in question is encrypted using LUKS, because the encryption system adds its own header to the start of the disk/partition and offsets all data towards the end, while the order of the sectors is left untouched. The workaround is pretty simple!

Let’s assume the device is /dev/sda, with at least one (encrypted) partition inside.

cryptsetup luksDump /dev/sda1

will dump the header information to the terminal, including the “payload offset” in LBA units. Now, if you want to determine the position of the bad sector inside the LUKS drive, you add the starting sector of the affected encrypted partition to its payload offset. Use that number for the “S”-parameter in the first example of the tutorial. From there, continue just as described.

Remember to use the mapped drive (e.g. /dev/mapper/sda1) for all filesystem related commands in the badblocks tutorial. If “fdisk” can’t show you the starting sector of the mapper device partitions, use “parted”:

select /dev/sda1

unit s

print

quit

I double-checked this method by dumping the sector data before and after the change (LUKS and physical) and comparing the contents with a hex editor – the calculation seems to be spot-on.

Another little piece of advice: After determining what damage will occur inside the encrypted partition, zero the sector ONLY on the physical drive and NEVER inside of the encrypted partition. LUKS will encrypt the zeroes to random gibberish, which will sometimes not trigger the sector remapping procedure of the drive. If you zero it on the physical drive instead, it will again be overwritten with valid data the next time you modify its encrypted version.

Also, if you are using a CBC encryption mode and zero the affected 512 byte block on the physical drive, a whole filesystem block (usually 4096 bytes) inside the LUKS partition will become corrupted, caused by CBC’s chained sector encryption scheme. To circumvent this from the start, use the newer LRW or XTS modes.

There IS one downside though. If a defective sector alters the encryption header and there is no header backup or duplicate of the contents – well, then that’s just that. With the header gone, there is no way to restore the contents of the drive! Same thing for your repair attempt – triple-check all dd-commands for typos before executing (especially if you are correctly using -skip or -seek)!

Adding S/P-DIF to your soundcard

This is something really simple I did a while ago…but wanted to show you nonetheless.

Most current notebooks still don’t provide an optical or even coaxial digital sound output, also known as S/P-DIF. This is something I can’t really relate to, as it might well be the only way to get encoded surround sound out of that thing without using an extra USB sound card or some crappy stereo upmixer. HDMI is also not the key to the problem since the digital audio stream is tightly embedded into the rest of the signal – an extractor circuit costs about 300 Euros over here!

Instead, save that time and take a look at your soundchip. Most feature S/P-DIF (especially if the containing device features HDMI) natively, but the corresponding pins are simply not connected to anything – which does not mean they are not active! Try to google for the part number (here ALC262), for most chips you will find datasheets right away. Once downloaded, head straight for the pin assignments section and keep an eye out for descriptors like “SPDIFO”.

Edit: If an output pin is available, chances are pretty good that it is useable even if the digital output is not shown in the mixer program. Some applications can turn off the optical output by software, but mostly that means the transmitter device while the data keeps on streaming.

In this case, the original docking station had the optical output built in even though the pin was left unconnected within the notebook, which made things a little easier – although it once again proves that manufacturers sometimes abandon pre-planned features for whatever reasons, which I as a customer generally don’t approve of.

Finding the corresponding pin on the dock connector was done using a generic multimeter. There should be no transistors or other obstructing parts in the signal path as most optical transmitters have built-in logic that only requires a TTL signal to switch the LED emitter. Out of the three or four pins on the transceiver device you should be able to identify one as ground, one as VCC and one of the remaining two will be the signal input. The remaining fourth one (if present) is most likely an enable pin and is tied to either VCC or GND.

Realtek ALC262 with upgrade
Realtek ALC262 with upgrade

Once the correct pin on the package is determined, carefully solder a thin enameled wire on top of it. This is the only tricky part and requires a thin tip and steady hands. Try not to heat it up for too long and check for accidental connections between the neighboring pins afterwards.

Aaaand you’re done. The signal you just tapped can be fed into an optical transmitter (e.g. TOTX… type) directly. By the way: If you are building/designing a digital to analog converter for audio purposes, this trick can be used to implement a very simple USB connection. Just get one of those cheap USB-plug soundcards. The chips inside mostly feature digital outputs and can be wired into an open TTL-compatible input of your DAC project.

You’ve got mail!

Two days ago, a small package arrived on my doorstep after 15 day’s worth of travel. Straight from China, 5 pc. of the L7251 spindle/VCM driver (remember the bad harddrive?), sold by Shenzhen Drivestar, who have a very friendly and competent customer service. Thanks again! I figured replacing the chip was the best option, looking at the chances of success.

L7251Well what do you know. After soldering in one of these, the little sucker spins again. It is not accessible, yet, but this is definately progress! Even the o’scope shows perfect waveforms at the motor terminals, but listening to the clicking and squeaking noises I’d say something is wrong with the heads.

After investigating a little further, I spotted the central problem: The drive has suffered a triple headcrash, meaning three of its trackheads have touched the disc surface prior to the fault and were literally ripped to shreds – but in the wrong direction! The head assembly was bent and got stuck near to the platter edges, *almost* in the standby rack. Something prevented it from getting there, as it should automatically move and lock there as soon as the disc loses power. My best guess is that the disc stack rotated against its normal rotation direction, which can happen if you carry the drive.

To explain: Normally, the heads would create their own air cushion using the movement of the platters and “float” over the surfaces during operation. While the platters are motionless, that cushion is not present and therefore there is friction between the surface and heads – and any motion of the platters is also directed at the heads. For that reason, the heads have to be parked before the disc slows down too far. Usually, some kind of mechanical tensioner handles that part. It also ensures that the head assembly never moves freely while the drive is stopped.

The resulting braking force during spinup probably caused the controller to fail, and after repairing that part, the disk spun and pushed the head out of the platter spindle.

To sum it up, it is very likely that the surface of the lower platters is severely damaged, and even if not – the only way to undo this is to mount a new head assembly. I will look into this a little further, but for the moment that’s just that until I get around to building myself a small “clean room box” for repairs on discs and displays.

Minor setback

Somehow I knew that this went over too easy. Fortunately, magic smoke SMELLS. Just got the battery out in time, but part of the damage already happened.

Defective LED
Bang!

Defective LED
...and another.

Some days ago, one of the triple-chains on the right half of the screen went out. While the darker area is clearly visible, the shadow is still illuminated by the neighboring LEDs. The cause is the LED in the second picture, the damage is only visible as a slight dark streak in the center of the yellow part. Then today the backlight fuse blew out of this world, and the reason is seen in the first picture. I knew this was a close call, but precisely “calibrating” the almost not existent isolation gap between LEDs and metal (by wedging a piece of paper in every here and there during glueing :-D ) gave me no no reason for concern. Maybe some flexing or thermal deformation did the rest.

Anyhow, got to rebuild this part. Not too bad though, the stripes are inexpensive and I wanted to include a diffusor sheet anyway. As good a chance as any I guess.

Edit: By the way, the driver circuit survived the whole mess just fine. That IC is some tough design! Haven’t managed to fry a single one of those so far.

DIY LED TFT Backlight

Today I finally got around to finishing my first design for a switching  LED display backlight driver. The circuit is built around a LT3518 switch mode driver IC working in buck (step down) mode in this application.

LED driver pcb topside
LED driver pcb topside

LED driver pcb bottom
LED driver pcb bottom

Someone figure out why SMD hand-soldering always looks ugly on macro images…but trust me on this, the circuit works like a charm ;-) The flux residues don’t hurt performance, but you really do want to keep any solder-balls or whatever metalliccy is floating around your circuit away from the tight parts.  If necessary, rinsing it with alcohol while carefully assisting with a brush does the job, followed by a soft coating of plastic spray to prolong the circuit’s life.

My design was done rather quick and dirty because it needed to get done. Just don’t expect THE most absolutely high frequency capable layout, though I guess that prominent airwire in the first picture crashes this illusion anyways. Consider this more of a first prototype to check out what this chip has up its sleeve in buck mode after already trying boost mode on Tobi’s LED driver, which will also be documented later on. So far, it seems to keep up with my demands quite well, it will be modified and optimized later on.

Measurements (all taken at maximum brightness with a tRMS multimeter):

V in I in V out I out P in P out Efficiency
Mains 20 V 0.25 A 9.6 V 0.400 A 5.0 W 3.84 W 77 %
Battery 11 V 0.41 A 9.6 V 0.400 A 4.51 W 3.84 W 85 %

Higher efficiency on battery is just perfect, even though it is barely into the operating voltage range. The measurements made on the go, so the numbers might not be absolutely accurate. If there is time, the scope will clear that up. Also, keep in mind that battery voltage will drop some over time. According to standard Li-Ion specs, it ranges from 12.75 V charged to 9.3 V discharged (although I have never seen it go below 10.5 V so far).

The schematic and board layout will be available in the next few days in EAGLE format, together with more photos of the “implanted” driver. Need to clean up the parts naming and stuff first.

While I am on it, enjoy my wicked hand drawing skills :-D

Prelim. schematic
Prelim. schematic

Layout description
Layout description

A few things to add to the drawings: First, the schottky-diode is of SK34A type, secondly the ceramics are all of X7R-rating type (refer to datasheet of  the LT on this one) and thirdly the connection between TGEN and CTRL/VREF ist also an airwire, which can clearly be seen dominating the circuit in the pictures and is unbelievably oversized.

This article will be followed by some pictures of the LED array and driver mounted inside the display. The former was already installed before I started this blog so I didn’t think of taking some shots at the time. It is nothing spectacular really, some chip-on-board (COB) slimline LED modules bought over at led-tech.de, soldered together and cut to correct length, then glued into place instead of the CCFL with thermal conductive glue.

NOTE: While testing the device to its limits, I am experiencing some problems at very low system voltage. When on battery only, the notebook’s internal supply lines drop to around 10 to 12 volts as opposed to rock-solid 20V when the mains is connected.

This driver was designed with up to 20V and only down to about 12V input in mind, which means a maximum of 80% switching duty cycle at low voltage. I figured that the internal supply lines would be thoroughly regulated but was mistaken as it seems. As soon as the voltage drops a quantum too far while powered only by batteries (eg. more power is drawn by any device), duty cycle rises over some magic number around 96% at 250kHz (see data sheet) and flickering occurs as the IC loses control over the current. This is why there certainly will be a v2 of this, and it will be buck-boost-mode to handle low voltage situations even better. I might also want to fix minor problems with shutdown mode still allowing a very small current through the ICs internal switch, but in comparison this one is totally unimportant.

Right now the v1 driver is successfully implanted in the screen bezel and works nicely. Screen brightness even beats the original CCFL backlight, though the lower screen border is not absolutely uniformly lit. I could care less, thanks to the windows taskbar using up that space. The notebook is a SX65S by the way.

Harddisk troubles

Somehow I managed to get a little work done last night (2AM), so here’s the story about the harddrive:

Some time ago, a friend dropped a dead 1TB harddisk on my workbench. It was installed in an brand new external disk enclosure and had signed off as soon as it was loaded with data. Eeek!

His first attempt was to write to the manufacturer of the drive and ask for a replacement PCB – some companies do it and in most cases the PCBs can be simply exchanged if the revision numbers are identical (they MUST be if you don’t want to risk even more data loss). In this case, they only offered him to exchange his drive for a new one. That would have been fine except for the data being lost.

So, I offered to take a look at the device. After plugging it in for the first time, I immediately noticed that the BIOS could not recognize the disk at all – it even hung during disk ID-ing, which is a strong sign of the drive controller not grasping what the heck is going on anymore. Meanwhile, the disk itself made beeping and whirring noises, followed by sharp clicks. As I could not hear the distinct noise of the spinning discs, I figured the spindle motor controller was a nice place to start off.

Current harddrive PCBs (WD10EACS in this case) consist  of two main function groups which are fully integrated into generic ICs or ASICs, meaning application specific integrated circuits.

PCB of WD10EACS
WD10EACS circuit board

The ICs in the red area are the buffer and the main controller. This chip contains the whole intelligence of the disk. SMART programs and everything regarding data organization or transfer runs in here. The buffer IC temporarily stores the data being written/read while it has not been processed by the head mechanism or sent to the pc. I also marked some other important stuff on the PCB along the way.

The green area is what makes your disk spin. The spindle/VCM driver IC (STM SMOOTH L7251 3.1) generates a three phase motor drive signal for the platter spindle and moves the arm according to the main controller’s wishes. You can find a datasheet on the web, but it is for the predecessor L7250, which is similar in function but not in its pinout.

Now, what most people don’t realize is, how complicated harddrives are. They have full onboard diagnostic programs (which are VERY poorly documented of course, and these well-protected secrets are what makes data rescue companies so special) and even though they look simple on both outside and inside, they have to be precisely calibrated for optimal performance. Hence the need for exactly matching PCBs. But, since this case has all indications of hardware failure, no diagnostic program will fix the damage.

Motor drive testing in this case is best done with the drive board unscrewed from the disk. This will eventually increase the “failed start” SMART counters in the disk’s long-term memory, but whatever – it’s busted anyway. The reason for unhooking the board is that the motor coils, if not defective, present unknown resistances and inductances between the drive outputs and make your measurement extremely difficult. After connecting the four probes of my digital scope to the spindle connector, I started sampling and plugged the board in…

Scope screenshot for HDD drive signals
Scope screenshot of motor phases

…and this is what happened. Excuse the poor contrast, I usually only use white backgrounds when printing, to save toner.

What you see is the main startup algorithm doing its job. The controller first tries to find out what position the rotor magnet in the spindle motor is in, so it can generate matching signals for rapid acceleration. It does that by applying voltage to the different coils in changing combinations, following a pre-programmed pattern. After measuring the current rise-time for each combination, it can calculate the magnetic influences in the motor coils and from that the actual motor position. To explain the different traces, yellow blue and green are the actual three phases and the magenta-colored trace shows the center tap which is not present right now, since the motor is not connected. The phases are switched against the rail voltages by MOSFET half-bridges integrated into the IC.

I marked the important part in the graph – the yellow phase is missing something on the top of its waveform – there should be short high-pulses like on the other two. The controller notices that and tried to restart the process at the red mark. After failing twice, it decides to try and spin the disk some. This is called coarse drive mode, the visible pulsing is not meant to spin the motor to a certain frequency but rather to just turn it in case the motor is “stuck” in a non-discernible state (even though that should never happen). This part of the signal is responsible for the whirring noise as the motor actually moves, but poorly so because of the missing high level on one phase.

The next step will be to either find an exchange for the driver chip since I can’t find a matching PCB, or find  a way to replace the internal MOSFET bridge of the drive with an external one. If anyone knows where to get the actual L7251 3.1 datasheet, I’d be grateful for a hint.