• Categories

  • Archives

  • On redirects, spiders and security

    Published July 13th, 2010

    Recently I’ve been working with an agency that has its own simple PHP web site framework. During the course of working with them, a problem arose: pages were disappearing, apparently without human involvement.

    With some detective work they had discovered that somehow the ‘secure’ CMS part of the site — where the client can log in to make changes — was being crawled by automated search engines. Part of the CMS is a list of all the site’s pages, each with links to the usual operations — edit, delete etc.  When the spider was indexing the pages, it also accessed the delete link, thereby deleting the page (much like this DailyWTF story).


    Anyway I took a look and while the security wasn’t great — it was based around cookies with no server-side validation — it still seemed odd that the spiders were able to access the pages. I implemented a slightly more robust system using sessions, added an entry to robots.txt, and marked it as solved.

    And then it happened again.

    I couldn’t work out what was going wrong, so to stop it happening I converted all the delete links to forms. But it was nagging at me — how was it that the search engines were reaching the pages at all? Why weren’t they being rejected when the security script checked for a cookie and session?

    Finally, the penny dropped…

    The security check worked by looking for a valid session, checking it for a ‘user is logged in’ value, and if one wasn’t found then sending a redirect header pointing to the login page. Nothing unusual there. So what was going on?

    When PHP sends a redirect header, the browser says “OK, I’ll go to this other page” and the user’s none the wiser. But just because the browser is no longer listening, that doesn’t mean the script automatically stops running. In fact, unless you tell it to stop it just continues as if nothing had happened. Thus, the spiders were simply ignoring the header and receiving the page as if there were no security in place at all.

    The solution? Add an ‘exit;’ after the redirect header. Simple!

    Web fonts: the time has come

    Published May 28th, 2010

    A couple of weeks ago, I spotted a post on Slashdot about a company – fonts.com –  that was opening up its font library for use by web developers. I thought it was a pretty neat idea: it uses the new-ish @css-font functionality that works in most modern browsers – along with a bit of JavaScript magic – to allow you to, in effect, embed a font in a web page such that it can be used like any other standard font.

    No more image replacement, or clunky sIFR nonsense to get pages looking as your clients actually want them to!

    After that, I became aware that there’s more than one web site out there offering similar service. So here’s a run-down of those I know of — feel free to add a comment if you know of any that I’ve missed:

    • WebFonts – as mentioned above, over 2,000 fonts to choose from and more on the way.
    • Cufon – allows you to upload your own font files and then download them compiled for web use.
    • FontFace – download pre-packaged ‘kits’ or roll your own.
    • Google Web Fonts – yes, even the mighty Google is getting in on the act, although the choice is rather limited.

    Update: From the comments (thanks!):

    Seems like it’s a technology whose time has arrived.

    PayPal Error: Sorry, an Error Occurred After You Clicked the Last Link

    Published March 10th, 2010

    I’m just coming to the end of a project that required checkout via PayPal. The system is to be hosted in an iframe embedded in a third party site’s page, but when redirecting to PayPal the iframe ‘pops out’ to take over the whole page (PayPal forces this behaviour).

    All was working smoothly using the ‘PayPal sandbox’ testing domain, but when it came to testing a live transaction in Internet Explorer I received the following error:

    “Sorry, an Error Occurred After You Clicked the Last Link”

    …followed by a lengthy but rather unhelpful block of text suggesting various courses of action — none of which was any use at all.

    Consulting Google, it seems that this is far from an unusual occurrence but there’s little consensus as to the cause. In the end, I tried adding target=”_top” to the form that submits to PayPal (thus forcing the ‘pop-out’ of the iframe), and presto, all is now fine again.

    CSS: Removing that dotted line around a link

    Published December 4th, 2009

    Just a quick note to myself…

    I’m working on a navigation system that uses CSS image replacement. To hide the text, CSS shifts it out of view by using the text-indent property and a big negative value. However, a side-effect of this is that when you click on a link, the dotted border that appears while you’re holding the mouse button down goes off the side of the page.

    To get around this, I just wanted to disable it. After some searching I discovered the outline property — setting this to ‘none’ gets rid of it entirely:

    outline: none;

    Of course, the above will remove it for all links, it’s up to you to decide whether you want to make it more specific.

    Converting latitude and longitude coordinates to map x and y values

    Published May 19th, 2009

    I’m currently working on a project to convert an existing Flash map, which uses location data that’s been manually entered and maintained, to use data that comes from a database and uses latitude and longitude values. The original data is stored in an XML file so updating it dynamically is not a problem but as you’ll know if you’ve ever tried it, plotting latitude and longitude values on a map isn’t as simple as it might sound.

    Briefly, the problem arises from the fact that the world is a sphere and representing it in two dimensions is not a straightforward task. There are various ‘projections’ to choose from, each of which uses a different method for mapping points — you can find out more in this Wikipedia article. The most common choice is the Mercator projection, where lines of latitude are spaced equally, and fortunately for me it’s the type of map used in the project I’m working on.

    So, being somewhat mathematically inept, I decided that Google would save me from a day’s head-scratching. That turned out not to be the case, as I tried a good half-dozen different suggestions before finally stumbling across one that actually produced accurate results. To save others from the same fate I decided to reproduce here the code I ended up with, which is based heavily on code found in this newsgroup post. My implementation is PHP, but it shouldn’t be hard to convert it to another platform (see the original thread for a Javascript implementation). Apologies for the rubbish formatting, WordPress isn’t great at handling code:

    function LongitudeToX( $lat, $lon, $map_zoom, $scale_value, $x_offset = 0 )
      $radius=$offset / pi();
      return ( ( ($offset+$radius*$lon*pi()/180)>>$map_zoom ) * $scale_value ) + $x_offset;
    function LatitudeToY( $lat, $lon, $map_zoom, $scale_value, $y_offset = 0 )
      $radius=$offset / pi();
      return ( ( ($offset-$radius*log((1+sin($lat*pi()/180))/(1-sin($lat*pi()/180)))/2)>>$map_zoom ) * $scale_value ) + $y_offset;

    A few notes about using these functions:

    • From what I can tell, the above code is based on Google Maps and the $map_zoom relates in some way to the ‘zoom level’ of the map. I set it at 15 and it seemed to work OK.
    • Because the dimensions of your own map might not exactly match the assumed dimensions of the map used in the calculation, the $scale_value parameter allows you to adjust the output to fit more precisely. I had to specify two decimal places of scaling to get an exact fit.
    • Finally, the calculation assumes that the ‘origin point’ of the map (x=0, y=0) will be at the top, left-hand corner. This isn’t the case on the map I’m working on (the origin is at the centre) so I included $x_offset and $y_offset parameters so that I could adjust the output values accordingly.

    In order to find the best values for the above parameters, I manually placed three widely-spaced temporary markers on the Flash map marking points whose latitude and longitude I already knew. Then I had the code render these points and adjusted the various parameters until the position of the generated points exactly matched the manually-placed markers.

    I hope this code helps someone avoid wasting the sort of time I did yesterday!