Scargill

Archive for June, 2014

I’ve put this together to make it easy to ship a link to various people – the entire world if need be until I get a road near our property fixed. Initially I was told the road was owned by Alnwick Estates, they then absolutely insisted that they’d sold it to Northumberland County Council NCC – and so it goes on – meanwhile the holes get bigger and bigger.

Here’s the location – just off the main street in Bellingham – see the pink circles… and here’s a couple of the more dangerous holes – they are by no means the only ones.

Filled up with water at night these are a danger to life and limb – not to mention car suspensions.  I’ve been writing now since the winter and such is the lack of energy of the departments I’ve dealt with – the holes are still there – despite pointing out that they are DANGEROUS.

Well, I’ve just found the email address for the highways agency – of course they probably don’t do ordinary roads, julst the likes of the A69 etc, but I can but try – I’m not getting anywhere with other people so far.   (‘ha_info@highways.gsi.gov.uk’) – here’s their website. http://www.northumberland.gov.uk/default.aspx?page=702

There are similarly dangerous holes on the road from Bellingham to West Woodburn, maybe 100 metres or more on the left just before you get to West Woodburn – we nearly took out our suspension one night in the wet – I reported the problem – nothing…..

I guess next stop the local newspapers?

I had a really interesting conversation this evening with a pal who’s into electronics as much as me.   I’ve been messing around with low-cost radio “mesh” networks which essentially give me the ability to read data and control outputs around the house wirelessly… 

For some time now, ever since the oil company tried to give us a mobile phone based oil sensor at which point I had to tell them they’d just screwed up for half of the UKs rural community for whom there are no mobile signals… I’ve been wondering how to make a safe remote reading mechanism for the oil tank.

Funny how you don’t put two and two together sometimes… there are available (from China as our lot are too greedy) some cheap ultrasonic sensors – like the ones you have in cars to help you not run people over.. for a couple of quid you get a little board with a transmitter and receiver and you can take the signal and get accuracy of sub-centimetre bouncing off stuff for distance measurement.  It occurred to me tonight – why not bounce off oil!  So the next step was – how to get that info back to me – well I already have home control measuring temperature – modifying a board would allow me to return how full the tank was – so using temperature as the equivalent if it returned say 10 to 30c – that’s 20 values – that’s a tank accurate to 5% – good enough….  and finally how to power all of that – well – my friend suggested a super-cap – after all it only needs to wake up once an hour – take a reading, report back and go back to sleep – and a little solar panel to keep the supercap charged.

When the board powered up it would talk to my network – which up to now looks like it can reliably send data from unit to unit back to base – I’m looking at a test rig here for the network – I have a board in front of me – the “receiver” – two boards in the garden acting as relays – and one in the living room sending out incremental numbers as fast as it can. Right now I’ve just received 700,000 packages without issue and I plan to leave it overnight on test  – hoping to hell it does not rain!!!

Could be onto a winner here… would be nice to add that to the armoury of info I have available at the touch of a button on my phone and tablet….  but it’s late at night – something for another day.

Another one for the techies I’m afraid….

Do you need to turn mains lights on and off but want to do it from, say and Atmel/Arduino-type setup instead of the standard remote controls you get with plug-in-the-wall mains switches? Then this might be for you. This is a variation of a blog I just put up on WordPress – with my latest updates.

This article is about decoding radio remote controls using Arduino or similar hardware and a cheap 433Mhz radio receiver (£2 on Ebay for receiver and transmitter pair) – and then use that data to control the remote switch units using a similarly cheap transmitter – and was written to serve a purpose: NONE of the existing libraries I could find out there managed to detect all of my radio remote controls.

I’m interested in adding radio remote for mains control to my house control system which is basically an Atmel 1284 chip with Ethernet and radio network. You’ll find various articles on the subject at www.scargill.net

Although it is easy to make a remote switching unit that will be part of a network and control mains lights, it is overkill and it is also quite difficult to find dirt cheap mains to 5v supplies to go into one of the limited range of plug-in-the-wall project boxes along with a controller and a relay.

With that in mind I started to look at the many mains switch remotes available out there. These generally work using simple 433Mhz radio receivers and accepting a code which could be almost any length but is normally somewhere between 10 bits and 50 bits.. the encoding techniques vary. Generally speaking these do not use any kind of “rotating” encoding for security and simply use a block of data, repeated as long as you hold the remote button.

BUYER BEWARE: If you want to control lighting using plug-in-the-wall 433Mhz controllers you should be aware of two things…

[a] the cheap controllers you find in the likes of B&Q use a single frequency which is EASILY blocked by other, similar transmitters.  I found when experimenting with a radio link at 433Mhz that it was easily blocked just by holding a button down on a mains remote controller.

[b] I found a nice cheap set of these plug-in-the-wall mains switches at ALDI recently, 3 remotes and a handset for £10 – what a bargain – but then looking at the fine print – you “train” them by turning them on and pressing a handset button within a few seconds – and that’s fine.. but, I thought, what about power cuts? Sure enough – you have to re-train them every time. What idiot came up with that brilliant idea? Going around the house re-training remotes every time the power fails? Surely not?  But I tested them and that is how they work. They are boxed up ready to be returned – not “fit for purpose”. That is, to be fair, the first time I’ve come across this but if you order online – beware!

Ok, so what could be difficult the handset lets out a 433Mhz modulated signal when a button is pressed – and the mains blocks pick up this signal, decode it and turn the plug on or off – so all you have to do is read the signal coming off the handset and use Arduino (or PIC etc) existing libraries to record and play back, right? Wrong – I had at least two handsets that use a code which was so long the existing learning libraries could not handle them AT ALL.

Eventually after much reading and learning I decided to do my own thing.

So, easy enough, (both the transmitter and receiver which I’ve linked to below have ground, power and signal leads – nothing else – the receiver appears to have two signal leads but they are connected together – you need only use either one) assuming the output of the simple receiver is LOW on standby, press the remote and look for the start bit – capture the length of each change of state until the end?  Right?

Well, no… the package is sent over and over – so you have to look for gaps between packages. Easy? Well yes except for AGC – automatic gain control. When the receivers are receiving “nothing” they get more sensitive and in my office the spikes coming out of the output were horrendous. I tried ignoring narrow pulses with limited results then I twigged.. if you run the remote control near the receiver – the noise goes away as the AGC (automatic gain control) kicks in…

So – press the remote control button – THEN start scanning – and store the gaps between pulses – say, 300 of them.  I did this with some REALLY simple code looking for high and low using the micros() function in Arduino (millis() is way too course) and keeping the code tight by simply storing the actual value of MICROS() in a long array.. ok not efficient but for temporary storage it works and I can get consistent results.

I discovered that all the controls send repeated packages with a short absence of signal in-between. Given the noise issue, the first package often gets scrambled – the breakthrough came when analysing the stored values and realising the solution is to search for the first repeat – i.e. after the first BREAK –hence ignoring the first package – previous attempts to use that had yielded varying results at the start of the package. By the time the second package kicks in the AGC has adjusted and the noise has all but gone.

So this would give me an output like this typical one from a handset.

They vary of course in length (not from key to key in that case just the order of bits varies) but from handset manufacture to another model or manufacturer).

What I did notice were pairs of zeros… ie pulses twice the width of others. On reading up this is called Manchester coding (or a variation) – I chose to ignore this – but knowing there will never be more than TWO different pulse widths helps a lot… in reality I realise that there is only one period in this type of encoding.. see this for pictures and explanation if you are interested. http://en.wikipedia.org/wiki/Manchester_code

As you can see from that link, the clock and data are encoded into the one signal.

But for my purposes it’s really NOT important to either understand or use this knowledge. All that is necessary is to send that stream out – fixed width per pulse as per the original and if there are two identical adjascent values (ie 00 or 11, detected by longer and shorter pulses compared to a fixed value indicated by the first pulse) then obviously you simply send the status quo to the output bit for twice the length of time.

But this uses up a lot of storage.  Looking at various handsets I discovered the largest signal seems to be around 40 or so bits… I made up a 12-byte array and wrote a routine to bit-shift the entire array by one bit – hence allowing me to convert the string above to a more sensible set of bytes.

In addition I added a byte at the start which represents the width of the basic pulse – as this could be as high as 1200us, I divided this by 10 to fit into a byte. In the example below you see 33 added to the start – ie 330us pulse width. On experimenting, that slight loss of accuracy seems unimportant and it saves a byte.

And so for any given button – we now have 11 bytes (in this case) to encode it. I decided that was GOOD ENOUGH.  A little routine to read from the end backwards, sending out pulses (or no pulse) for the duration specified in the first byte – VOILA I was accurately switching mains lamps on and off via my cheap transmitter.

Most of the code is unique, some bits are variations on stuff I found on the web (thanks to all who’ve taken the time to put their ideas up on the web) – and is provided here UTTERLY without support – no comments on styling please – if you’re having trouble reading radio remotes for your purpose – I hope this is useful.

The transmitter and receiver are here

http://www.ebay.co.uk/itm/433Mhz-RF-Wireless-Transmitter-Receiver-Set-NEW-/131093418464?pt=UK_BOI_Electrical_Components_Supplies_ET&hash=item1e85c6d1e0

Change input and output pins to suit your project – any pin will do, if you get nothing having pressed your remote then sent A to the board via the Arduino serial monitor, then try messing with INTER_CMS-MS and MIN_POS_WIDTH but I suggest not before you do that, you check with a scope that something is actually coming in.

Clearly in any end gadget you make for your use, you’d not include the learning part or it’s huge TBUFF array. When I get a minute I plan to come up with a means to simply store the data in EEPROM and read from that…  mind you – if you use a 1284 chip instead of the old 328, memory isn’t quite the problem.

If you need more range, you could consider putting the transmitter (NOT the receiver) on a higher voltage up to 12v. I tested mine at 12v and the signal from the 5v logic still worked perfectly. The increase in range is noticeable – I’d say 50% better coverage.  If you blow up your board however, that’s your responsibility – don’t come back to me…

This is all suitable for short bursts to control stuff – don’t get any ideas about continuous home control signalling unless you can guarantee there are no other 433Mhz transmitters in use! I went down this route before realising it’s a non-starter and am now looking at the slightly more expensive frequency hopping chips for more sophisticated use – but if all you want to do is turn something on and off – this could be for you.

So – two programs – the first grabs the radio output and puts it up on the serial –it also has a test mode – the second is playback only…

Here’s the first with some test options…

// I used a simple 433Mhz tx and RX pair - 1 pin each needed. Output of radio assumed 1 when something there ie active high
// No interrupts or clever tricks used
// Using F() in strings merely to keep RAM down, and PROGMEM for arrays - that's the data you see on the serial display for one particular button press... 
// Could store this in EEPROM or whatever... for now could get a lot of these for a particular application into ROM without using up precious RAM
// None of the PROGMEM stuff is essential - just there to minimise RAM use
// Coding Peter Scargill June 2014. www.scargill.net

#define PIN_DATA_IN 2
#define PIN_DATA_OUT 3
#define SERIAL_SPEED 115200
#define INTER_CMD_MS 100
#define CMD_REPEATS 5
#define COMPRESSED_ARRAY 12 // assume that's enough
#define MIN_POS_WIDTH 200 // used before we have the period - assumed minimum acceptable PERIOD less than which we ignore - in Microseconds
#define MAX_NEG_WIDTH 2000 // assumed maximum acceptable ZERO state in uS above which is assumed a gap between blocks of data
#define READINGS 300 // For temporary storage while learning, need 4 bytes per reading... so WATCH OUT - 328 only has 2k of RAM
#define TIMEOUT_VALUE 3000000 // Number of microseconds before we give up the ghost - ie 3 seconds - likely won't happen due to noise

// Byron reports 39us*10 for period - didn't work - so I put in 59 - works a treat.

const byte g0[]PROGMEM={59,77,211,36,73,146,52,73,210,52,1}; // Byron D2on
const byte g1[]PROGMEM={59,217,150,109,217,178,45,91,18};
const byte g2[]PROGMEM={59,219,182,109,217,178,45,91,18};

byte tbuf;

int remno=0;

byte arr[12];

// Used to set a timeout
unsigned long start_time;
unsigned long timeout;
unsigned long current_time;

unsigned long calc;

// Stores all the values read here
unsigned long readings[READINGS];
int current_reading = 0;

// Collect serial data into an array - or timeout as you prefer
void readRadioPulses()
{
    for (int a=0;a<COMPRESSED_ARRAY;a++) arr[a]=0; // wipe the eventual storage array
    for ( current_reading = 0;  current_reading <READINGS; current_reading++) // store enough values for 2 or more packages - as long values for speed
    {
        // Waits until the input goes HIGH 
            while (!digitalRead(PIN_DATA_IN))
            {
                if (micros() > timeout) return;
            }
        // When the first value comes, save that period
        readings[current_reading] = micros();
        digitalWrite(13, HIGH);
        // Wait until the input goes low

            while (digitalRead(PIN_DATA_IN))
            {
                if (micros() > timeout) return;
            }
        current_reading++;
        readings[current_reading] = micros();
    }
}

void shiftArray(int x) // move the entire array toward the highest offset by 1 bit and add least new significant bit to offset 0
{
  for (int a=sizeof(arr)-1; a>0;a--)
    {
      arr[a]<<=1; arr[a]|=(arr[a-1]&128)>>7; 
    } 
  arr[0]<<=1; arr[0]|=x;
}

// Sends the data to the serial port
void displayRadioPulses()
{
  int speed;
  byte pickone=0;
 
  for ( current_reading = 0;  current_reading < (sizeof(readings)-1); current_reading++)
    {    
      calc=readings[current_reading+1]-readings[current_reading]; 
      if (((current_reading&1)==0)&&(calc<MIN_POS_WIDTH)) { current_reading++; continue; }      
      if ((calc>MAX_NEG_WIDTH) &&((current_reading&1)==1)) { pickone++; if (pickone==1) speed=readings[current_reading+2]-readings[current_reading+1]; }
      if (pickone==1) // extract string here
          {
          if ((current_reading&1)==0) 
              { 
              if (calc>(speed+(speed/2))) 
                    {
                    //Serial.print("1"); 
                    shiftArray(1);
                    }
                //Serial.print("1"); 
                shiftArray(1);
              } 
              else 
              {
                if (calc>(speed+(speed/2))) 
                  {
                    //Serial.print("0"); 
                    shiftArray(0);
                  }
                 //Serial.print("0");  
                  shiftArray(0);              
              } 
          }
    }
 // Serial.println(""); 
  Serial.print("const byte g"); Serial.print(remno++); Serial.print("[] PROGMEM={");
  Serial.print(speed/10); Serial.print(",");
  for (int a=0;a<sizeof(arr);a++) { if (arr[a]) { Serial.print(arr[a]); if (arr[a+1]) Serial.print(",");  } }  Serial.println("};");
}

void sendRadio(const byte *m, int sz) // can't do sizeof a PROGMEM array so just pass the size
{
   int scount;
   int offs;
   int period;
   byte tmp;
   period=(int)pgm_read_byte(m)*10; // get the first value, multiply by 10 and that is your period
   for (scount=0;scount<CMD_REPEATS;scount++)
     {         
       for (offs=sz-1;offs>0;offs--) // start at the end and work backwards not including offset zero which is the (period/10)
          {
          tmp=pgm_read_byte(m+offs); Serial.print(".");
           for (int r=0;r<8;r++) { if (tmp&128) digitalWrite(PIN_DATA_OUT,HIGH); else digitalWrite(PIN_DATA_OUT,LOW); start_time=micros()+period; while (start_time>micros()); tmp<<=1; }
          } 
          Serial.println("");
        digitalWrite(PIN_DATA_OUT,LOW); delay(INTER_CMD_MS);
     } 
}

// Listens to SERIAL for inputs A=read, B=write demo - when reading press and hold handset FIRST
void SerialRequest()
{
   char c;
   while (Serial.available()==0);
   c=toupper(Serial.read());
    if (c=='A')
      {
        start_time = micros();
        timeout = start_time+TIMEOUT_VALUE;
        readRadioPulses();
        displayRadioPulses();
        Serial.flush();
      }
      
      if (c=='B') // try the example compressed code which includes (period/10) as first byte - can be any length
      {
       sendRadio(g0,sizeof(g0));
      } 

     if (c=='C') // toggle 2 codes back and forth for range testing
      {
       Serial.print(F("Test toggling..."));
       for (int a=0;a<200;a++)
       {
         sendRadio(g1,sizeof(g1));
         delay(1000);
         sendRadio(g2,sizeof(g2));
         delay(1000);
       }
       Serial.println(F("Done"));
      } 
}

void setup() // set serial speed, set output for the radio output - and put something on screen to show it's working - use built-in F() to save RAM
{
    Serial.begin(SERIAL_SPEED);    
    pinMode(PIN_DATA_OUT, OUTPUT);    
    digitalWrite(PIN_DATA_OUT, LOW);
    Serial.println(F("Send A to check or B to send example in PROGMEM"));
}

void loop() 
{
    SerialRequest(); // I could have just put the lot in here really
}

and here is the second – playback only…

// I used a simple 433Mhz tx  - 1 pin each needed. Output of radio assumed 1 when something there ie active high
// Coding Peter Scargill June 2014. www.scargill.net

#define SERIAL_SPEED 115200

#define R433_PIN_DATA_OUT 2
#define R433_INTER_CMD_MS 100
#define R433_CMD_REPEATS 5

// Byron reports 39us*10 for period - didn't work - so I put in 59 - works a treat.

const byte g0[] PROGMEM={64,77,211,36,73,146,36,73,210,52,1};  
const byte g1[] PROGMEM={64,73,211,36,73,146,36,73,210,52,1};
const byte g2[] PROGMEM={64,77,211,36,73,146,52,73,210,52,1};

void sendrad (const byte *code_buffer, int sz) // can't do sizeof a PROGMEM array so just pass the size
{
   int scount;
   int offs;
   int period;
   byte enc;
   long mydelay;
   
   period=(int)pgm_read_byte(code_buffer)*10; // get the first value, multiply by 10 and that is your period
   for (scount=0;scount<R433_CMD_REPEATS;scount++)
     {         
       for (offs=sz-1;offs>0;offs--) // start at the end and work backwards not including offset zero which is the (period/10)
          {
          enc=pgm_read_byte(code_buffer+offs);
           for (int r=0;r<8;r++) 
             { 
             if (enc&128) digitalWrite(R433_PIN_DATA_OUT,HIGH); else digitalWrite(R433_PIN_DATA_OUT,LOW); 
             mydelay=micros()+(unsigned long)period; 
             while (mydelay>micros()); 
             enc<<=1; 
             }
          } 
        digitalWrite(R433_PIN_DATA_OUT,LOW); delay(R433_INTER_CMD_MS);
     } 
}

// Listens to SERIAL for inputs A=read, B=write demo - when reading press and hold handset FIRST
void SerialRequest()
{
   char c;
   while (Serial.available()==0);
   c=toupper(Serial.read());
   switch(c)
   {
      case 'A' :  sendrad(g0,sizeof(g0)); break; 
      case 'B' :  sendrad(g1,sizeof(g1)); break; 
      case 'C' :  sendrad(g2,sizeof(g2)); break; 

      default  :  break;    
   }
}

void setup() // set serial speed, set output for the radio output - and put something on screen to show it's working - use built-in F() to save RAM
{
    Serial.begin(SERIAL_SPEED);    
    pinMode(R433_PIN_DATA_OUT, OUTPUT);    
    digitalWrite(R433_PIN_DATA_OUT, LOW);
    Serial.println(F("Send A to N to send examples in PROGMEM"));
}

void loop() 
{
    SerialRequest(); // I could have just put the lot in here really
}

Here I am in sunny Brussels again, this time for two different bodies, I came here at the start of the week to continue the work on ESCO with colleagues from around Europe – and I end the week at Microsoft HQ here in Brussels.

I’ve been staying at a very weird hotel called the Mozart near Grande Place – and I mean 2 minutes near. The rooms are a tad small but as you can see below they make up for it in sheer elegance – the whole place is done out like this…

I arrived Monday evening in time to meet my friend Kenneth from Sweden at the local bar, before heading into 2 days of intense meetings which finished last night. I headed back to my hotel and had a drink outside with another colleague George and we talked about data security which is his field – it really didn’t occur to me before but all this friend “tagging” we do on photos is giving the security forces tools they never before dreamed possible! The evening ended on a fine note as I met up with a friend who long ago was the FSB’s Policy Chairman, Brendan Burns (and now a member of the EESC) and he introduced me to a fine Thai restaurant a short walk down from the square – he has a flat here and spends a lot of time in Brussels so I always learn a lot of local info when we meet up.

Today I’d planned a trip to the Atomium and thereabouts but I simply didn’t manage to get out ofbed early enough and so decided to do the up-hill walk to Avenue Des Arts to attend a meeting put together by ESBA and SEAP to discuss the Transatlantic Trade Agreement – met some interesting characters there including one who gave me a whole new way to think about open source and who’s views on the future of 3D printing are similar to mine.

We had the American Ambassador to the EU there as well as a host of other relevant people and there was a very lively debate on the subject, including an amazing attack by an MEP who came over initially as quiet and announced he had no idea why he was here – then promptly launched a vicious and somewhat effective attack AGAINST the TTIP.

Great afternoon and many of the guys are meeting up tonight for a meal.

Tomorrow an AGM (I’m merely an observer in this case) then back home to see my wife and her broken foot.

Ok, Brussels is a big, noisy city – but given that, this is a nice place to be….back here in a month for another round of ESCO meetings.