The Design and Implementation of the BOXER Network Compiled by Aztech -------------------------------------------------- (Culled from 'British Telecommunications Engineering', Vol 13, Oct 1994) From what I can determine from this article and the accompanying schematics and documents,this looks set to mirror AUTOVON in the UK and Europe. However,whereas AUTOVON can be accessed via the PSTN with a simple extended DTMF dialset,this looks like it can only be accessed by direct dialling-in or by radio patching.Also,this Network looks likely to be carrying a high data ratio,due to the amount of cash being lavished on the Intra-Node links. Eavesdropping is out of the question,as the system will no doubt be implementing the TEMPEST security measure.For more info on TEMPEST and its implications,read the end of this file. ***** ***** ***** ***** ***** ***** ***** ***** The BOXER Network,one of the largest private networks in the UK,provides nationwide connectivity to RAF and other military and government users. This article describes various aspects of the network,which has been designed and implemented by Syntegra,BT's systems integration buisness against exacting Ministry of Defense requirements. INTRODUCTION ------------ The BOXER Network has been designed and implemented by a dedicated project team within Syntegra,BT's systems integration buisness,against a Ministry of Defense(MoD) contract placed in 1987,with a scheduled completion and final handover date to the RAF of June 1996. The BOXER Network provides full nationwide connectivity to RAF and other military and government users as part of one of the largest private networks in the UK.The Network forms the transmission element of the RAF fixed telecommunications sytem(RAFFTS).As a military network the system has been designed to meet the specified requirements of security and survivability. BOXER consists of over 200 individual sites,the majority of which are further broken down into subsites.The basic topology is shown in Figure 1. As a standalone MoD network,one of the primary design aims upon BT was diversity from existing BT sites wherever possible.BOXER uses optical-fibre and microwave radio systems to provide the bearer network,and operates at hierarchical data rates up to 140 Mbit/s.A general outline of the Network is shown in Figure 2.The Network consists of over 4.5M channel kilometres of 64 kbit/s digital paths. One of the first decisions made was how the project should be organised,given the size,complexity and geographical spread of the network.It was concluded that the use of a phased approach,with each phase consisting of six month periods,provided the most manageable solution.Using this as an objective,the project was broken down into specific design/implementation elements,the functions and operations of each being as follows. NETWORK DESIGN: --------------- The project team recruited Network radio and cable planners from what is now BT Worldwide Networks to carry out the the detailed design work. The design of the microwave network used the experience built up by BT research at Martlesham Heath for the main BT network,and relied on the proven computer aided design packages developed there.Among the software packages used heavily is the Peacemaker path profiling and interference study suite of programs.Having completed desktop studies using these systems,detailed route and site surveys are undertaken in order to confirm the equipment configurations to be supplied,tower availabilty and antenna mounting access at the required heights. Cable planning operates in two major areas,on site and access.The on-site cable planning identifies the routes required between the microwave radio access site and that of the end traffic user(see figure 1). FIGURE ONE : BOXER TOPOLOGY --------------------------- RADIO TESTABLE SECTION SUB SITE *{======}**{===============}* * ** * +---BOXER TPB---+ * ** +---+ * +--------------+ | +--+ +--+ | * ****| | * | +--+ +--+ | | | |*| |********* ****| | *********| |*| | | | +--+ +--+ | +---------------------+ | +--+ +--+ | | * | | HILL TOP | +-------- * ---+ +----*----------+ * * RADIO REPEATER * * * * CROSS-SITE TIE TESTABLE SECTION * * * +----*----------+ +-------------*-----+ | * | | * | | +---+ | | +---+ | | | | | | | | | | +---+ | | +---+ | | SWITCH | | +---+ | | +---+ | | | | | | | | | | +---+ | | +---+ | | * | +----*----------+ +-------------*-----+ * * +---+ FIBRE REPEATER +---+ | |******************><***********><*********************| | +---+ +---+ OPTICAL FIBRE TESTABLE SECTION RAF BASE STATION RAF BASE STATION TPB:TRANSPORTABLE PREFABRICATED BUILDING *** INDICATES PHYSICAL LINK === INDICATES RADIO/MICROWAVE LINK Access Planning concerns those routes where fibre provides the transmission medium into the site via dedicated duct installed in public highways in a matter identical to BT and other utilities.As such,the requirements of the New Road and Street Works Act and its predecessor The Public Utilities Street Works Act(PUSWA)apply and need to be considered and implemented in addition to project specific requirements. The culmination of the above planning actitivities is the identification of all elements required to be provided on a site in terms of civil works,etc.in order for the project to be implemented.This information is embodied in a site-specific civil-works requirement document,which is issued to the MoD or the site owners by the project team with sufficient lead-time to ensure that works are completed by the planned site implementation date. The final planning activity concerns traffic and network sizing: The overall network traffic requirements and connectivity are the responsibility of the RAFFTS planning team;the physical implementation in terms of capacity and multiplexing lies with the BOXER project team.The results of this planning effort are specific multiplex and equipment plans for each site in the network;these are used to define the equipment fit for each location (see later). EQUIPMENT: ---------- The BOXER Network utilises over 30 major items of equipment ranging from power supplies including stand-by generators and rectifiers,transmission equipment(Multiplexers through to Microwave Radio Systems) and equipment housing.Of these the most important is the Transportable Prefabricated Building(TPB).This is the cornerstone of the project,as it provides security and survivability protection for the transmission and ancilliary items of equipment.The TPB in its final configuration provides enviromentally controlled power and equipment mounting areas seperated by a fire wall,each with its own means of external access.External Protection for the TPB is provided by insulated moulded weatherproof GRP panels,which are designed to blend with the location by means of various finishes:for example,brick or plain coloured as standard,or others to meet special requirements. The overall dimensions of the TPB(7m x 3m x 3m)allow for transportation by road,and the design is provided with suitable lifting points not only for craning on and off the low loaders for road delivery,but also for helicopter delivery to remote locations. The procurement of all items of equipment is based on competitive tendering exercises managed by the BT Group Procurement Services personnel allocated to the project.Each item of equipment required is based on standard commercially available items wherever possible to minimise specific develop- -ment programmes and products.These are then evaluated against project specific facility requirements contained in BOXER specifications.These documents,allied to specifications defining the BOXER project requirements in both the commercial and quality assurance areas,form the basis for the tender adjudication exercises. The procurement exercises operate under a strictly formalised methodology based on acheiving an overall value-for-money solution covering whole-life costs.These exercises resulted in over 500 individual tender responses to the various requests,and,after adjudication and acceptance by the MoD,resulted in contracts worldwide with various major telecommunication suppliers. IMPLEMENTATION: --------------- The size of the Network and the time scales required neccesitated a structured programme of implementation.This programme consists of three distinct elements: ELEMENT 1(RELEASE POINT 1) -------------------------- This element concerns the procurement and delivery of the individual equipment items from the manafacturers.Each equipment type used within the project is subjected to a specific type-approval exercise to confirm compliance with the network performance requirements.Based on these tests,a set of release criteria are then agreed with the companies;these criteria are used by the projects quality assurance (QA) personnel as a basis for acceptance of each batch of equipment.Based on their performance in this area,and to maximise effective use of the projects QA resource,companies are encouraged to achieve 'delegated release status' with batches of equipment being released under certificates of conformance issued by their own internal QA organisation. Delivery of the procured items directed to one or two specific areas:either the centralised installation facility (Element 2 below) or direct to site in case of items such as antennas and associated mounting steelwork. ELEMENT 2(RELEASE POINT 2) -------------------------- This element concerns the site specific configuring of the TPBs.In order to achieve this,a centralised installation facility was developed and operated by Fujitsu Fulcrum Communications Ltd. under a BT contract.This facility is required for two reasons: *The range of items to be installed in the TPB with the associated fittings and fixings precludes this as an on-site operation. *The installation rate required(an average of 44 TPBs per year)demanded significant manpower with limited scope for efficient utilisation given the wide geographic spread of sites. The manufacturing cycle through the centralised installation facility follows a standard sequence,which is now described. The configuration in terms of equipment fit is identified as part of the network design and planning functions of the main BOXER team. This information is then transmitted to Fujitsu Fulcrum.Fujitsu Fulcrum is responsible for designing and fitting out the TPB under the overall management control and supervision of the project. Initial fitting out includes the provision of overhead ironwork and trunking to support the TEP-I(E),CEPT slimline or other equipment housing requirements. AC electrical power wiring is then installed to complete the basic fitting of the building.The building is now ready for the equipment to be installed. Radios utilising CEPT slimline practise and the TEP-I(E) racks containing the multiplexes and optical line terminating equipment are fixed to the overhead ironwork. The power equipment consists of two primary elements,both based on standard BT products.The first,installed in the power room of the TPB,is the stand-by generator(PS 4006 equivalent) and its associated ventilation/exhaust system,and the second is the rectifier and stand-by battery system EP2008B installed in the equipment room.Once the physical installation of the equipment is completed,the remaining power and transmission cabling can be undertaken.When this is completed,the configuration and connectivity of the equipment elements via digital distribution frame(DDF) cabling is matched to the defined traffic connectivity requirements for the site the TPB is allocated to. The final element of the Fujitsu Fulcrum manafacturing cycle is testing(see later).The design of the Fujitsu Fulcrum facility allows for the benifits of a full factory manafacturing cycle to be implemented.This cycle takes 12 weeks to complete for each TPB,and at the production rate required for the project,12 TPBs will be in manafacture at any one time. This structured process allows for maximum throughput in a controlled manner,and provides for efficient utilisation of the available manpower resources and skills. ELEMENT 3(RELEASE POINT 3) -------------------------- The final implementation element is installation on site and associated testing/commissioning.BOXER sites are located throughout the UK mainland. Although Road access is acheivable at all sites,in several cases four-wheel drive or special purpose vehicles have to be used.In these cases,delivery of the TPB on its standard road low-loader transport can prove impossible: A helicopter or other speciaist delivery system is then neccesary. A fully equipped TPB weighs between 9 and 10 tons,which is just within the maximum load of an RAF Chinook depending on how far the building has to be carried from the access to the site.As stated earlier,the TPB is designed with appropriate lifting points to facilitate this delivery method,which has already been successfully utilised,and is scheduled for a number of other site deliveries before the project is completed. Each site must be equipped with appropriate mounting facilities for the TPBs.These foundations and associatied other civil works,for example, ducts etc..have to be completed before deliveries are commenced.The off-loading of the TPB,as is the mounting of antennas on microwave towers, is managed by BT Worldwide Networks via a service level agreement.Their responsibilities cover the off loading and safe bolting down of the TPBs,and the installation,panning and testing of the antenna systems. Once a TPB is bolted down,the responsibility for installation reverts back to project personnel.The building is unpacked and the task of recommissioning starts.This occupies amny functions:connection of the TPB to its primary 240v AC power supply,the connection of the stand-by generator to its fuel supply, fitting of weather-proof cowls over the enviromental ventilation grills, and re-installation of the transmission equipment modules/cards into their rack positions.These items are demounted and packed for transportation purposes by Fujitsu-Fulcrum before dispatch as a general policy,although tests proved that transportation of a fully equipped building is a viable option for most of the UK sites.Once the physical reinstatement of the TPB is complete,power can be connected to the equipment to allow testing to take place.The objective is for the primary power to be connected to the TPB such that the enviromental control systems (heaters/fans) are operational on the day of delivery;a stable enviroment for the implementation personnel will therefore exist during testing and commissioning. Once all the testing is completed,the building is cleaned and the internal cabling connections on the DDF are returned to the traffic design requirements before handover to the customer.Figure 3 shows a typical TPB installation. (Figure Three shows a photo of a BOXER node in action.The Building containing the communications equipment is seperate from the tower holding the Microwave Transmitter/Receiver.Judging by the cable thickness and the overall dimensions of the dish,it looks like a VERY powerful setup.-=[AH]=-) PERFORMANCE ----------- The performance of the network is assessed in two ways,firstly in terms of individual equipment elements and secondly in terms of the network itself. EQUIPMENT TESTING ----------------- The testing strategy adopted in the project follows similar lines to that for implementation:the concept of reducing structured testing through the three stages identified as Release Points 1,2 and 3 allows for minimum staffing levels and time on site. RELEASE POINT ONE ----------------- This testing covers the factory release of the individual items from the manafacturers.The tests are derived from those used for design acceptance, and are used by the project QA personnel to assess each company's performance against the company's quality plans.Formal acceptance of batches of equipment requires either on-site visits or acceptance of certificates of conformity issued by the individual company's QA authorities.For the microwave radio systems,these tests have to be carried out on equipment configured as 'hops' and incorporate both equipment and hop testing.This is made possible since equipment is ordered on this basis,and not on individual terminal ends. Under the contracts the manafacturers must hold full records of all the test results,as refernce sources for both BT and the MoD quality assurance departments,and are used as reference data for the regular audits identified in the agreed quality plans. RELEASE POINT 2 --------------- Two stages of testing take place at the centralised installation facility: goods inwards and goods outwards or final release. * Goods Inwards Testing Owing to the complexity and numbers of different items involved within the project,the need for initial testing upon delivery to the CIF to check for damage in transit and basic functionality is essential.On many of the items, for example,multiplexes and line terminating equipment,this utilises standard test-rigs;however,for the larger more complex items,particularly the radios, full assembly becomes a prequisite of testing.Because of this,Fujitsu Fulcrum modified the goods-inward testing strategy for the radios to incorporate hop testing as a more effective method of proving performance.Certain items of equipment can be functionally tested only after installation in the TPB, particularly the COMPAC stand-by generator.Goods-inwards testing of this and similar items is therefore limited to signs of visible damage.Having some degree of certainty that the individual items are functional,installation in the TPB then takes place. *Goods Outwards Testing This testing is undertaken once full installation of the equipment is completed.As identified earlier in this article,each TPB is configured on a site-specific basis equating to the traffic levels both through and terminal. Thus by means of the DDF,equivalent traffic paths can be constructed within the building to verify performance and interoperability between the equipment items.Looping at IF on the radios allows for at least partial verification of those items allied to single terminal testing of parameters such as output frequency,power and spectrum.With the genertaor installed in the TPB, along with its associated input and exhaust systems,verification that it will start and run satisfactorily can now take place.Also the wiring of the TPB is tested and certified as being in accordance with the IEE wiring regulations. Once all this testing is completed,a certificate of conformance is issued and the TPB is prepared for dispatch to site.As with Release Point 1 testing,all test results are held by Fujitsu Fulcrum to be used as a basis for ongoing project and QA audits. For those items of equipment delivered direct to site,for example, waveguide and antenna systems,optical-fibre cables etc.,the requirements for Release Point 2 Testing are waived. RELEASE POINT THREE ------------------- On-site testing is limited to that necessary to confirm that no damage has occured in transit for those items delivered from the centralised installation facility,to verify the performance for those items delivered direct to site,for example,antenna and feeder systems or installed optical-fibre cables,and to test overall radio hop/optical fibre sections. BT Worldwide Networks install and comission the antenna andwaveguide feeder systems and pan the links.Gas barrier-to-gas-barrier loss and swept-frequency return losses provide acceptance criteria for this work. Testing of the equipment within the TPB or end-user cabinets is carried out by the project implementation managers and engineers against a set of procedures defined and agreed with the MoD.Hop/fibre section testing is carried out against targets derived from the network performance objectives described in the next section,and these results form the basis for final acceptance by the MoD. FIGURE FOUR - BOXER HYPOTHETICAL REFERENCE DIGITAL LINK - - - - - - - - - - - - - - - - - - - - - - - - - - - - +-----+ *-----| LTE |-------------+ PRIMARY +-----+ | TRUNK SWITCH +----+----+ | LTE | +----+----+ | +----+----+ | 2/8 MUX | +----+----+ | +----+----+ | 8/34 MUX| +----+----+ | ------- +-----+-----+ ------- | |RADIO TERM | | | +-----+-----+ | | | | | +-----+-----+ | 100 km |RADIO RELAY| | 4.5 Ghz | +-----+-----+ | RADIO | | | | +-----+-----+ | | |RADIO TERM | | ------- +-----+-----+ ------- | +----+-----+ | 8/34 MUX | +----+-----+ | +----+-----+ | 8/34 MUX | +----+-----+ | +----+-----+ |34/140 MUX| +----+-----+ | ------- +----+-----+ ------- | | FOC LTE | | | +----+-----+ | | | | | | | | | | 75 km | | | FOC | | | | | | | | | | | | | +----+-----+ | | | FOC LTE | | ------- +----+-----+ ------- | +----+-----+ |34/140 MUX| +----+-----+ | +----+-----+ | 8/34 MUX | +----+-----+ | +----+-----+ | 8/34 MUX | +----+-----+ | ------- +-----+-----+ ------- | |RADIO TERM | | | +-----+-----+ | | | | | +-----+-----+ | 75 km |RADIO RELAY| | 15 Ghz | +-----+-----+ | RADIO | | | | +-----+-----+ | | |RADIO RELAY| | | +-----+-----+ | | | | | +-----+-----+ | | | RADIO TERM| | ------- +-----+-----+ ------- | +-----+-----+ | 8/34 MUX | +-----+-----+ | +-----+-----+ | 2/8 MUX | +-----+-----+ | +-----+-----+ | LTE | +-----+-----+ | +-----+ | *----| LTE |---------------------+ +-----+ FOC:FIBRE-CABLE LTE:LINE TERMINATING EQUIPMENT PRIMARY TRUNK SWITCH NETWORK PERFORMANCE ------------------- The BOXER network performance is assessed by using the principles defined in CCITT Recommendation G.821. With the mix of data rates between 2 and 140 Mbit/s,and the configuration of the network,sections of the 'high,medium and local grade' elements of the hypothetical reference digital link (HRDL) defined in G.821 need to be considered.By using G.821 as a basis,a specific BOXER HRDL (See Above) has been derived and agreed with the MoD and their supporting technical experts.From this,performance targets in terms of errored seconds and degraded minutes are derived on a per kilometre route basis.The performance targets for each link within the network can then be calculated relative to its path length. The network is considered as a series of testable sections(See Figure one) and performance is verified on each of these.Since each sections performance is calculated on a per-kilometre basis,chaining series of sections together to form logical links allows the overall end-to-end section objectives to be met.The acceptance of each section by the MoD as they are completed is based on the results of a final 200 hour stability run against calculated targets allied to confirmation of testing at Release Points 1 to 3 as defined above. In terms of the BOXER Network,overall performance lies between the 'high and medium' grade CCITT objectives;however,by planning the network solely against the 'high grade' criteria,the customer gains benefit in terms of reliability and performance. The performance of the network against the objectives is the subject continuous monitoring throughout the implementation programme by the BOXER team,and the RAF provides results on the transferred sections.So far, in annual assessments,BOXER has comfortably exceeded all its targets by a considerable margin. (The BOXER project is due for completion in June 1996.) ***** ***** ***** ***** ***** ***** ***** ***** The article then went on for about a page about the Managerial structure of Syntegra,and how clever they all were,and how it was such a wonderful idea,bleh,bleh,bleh..You get the picture. PROJECT TEMPEST,AND ITS PART IN THE BOXER NETWORK: -------------------------------------------------- More info on Tempest can be obtained from: Christopher Seline cjs@cwru.cwru.edu cjs@cwru.bitnet TEMPEST(Transient Electromagnetic Pulse Emanation Standard) is essentially an electronic bugging standard/implementation(?) which allows anyone with a suitably equipped CRT and Scanning/synchronization equipment to receive visual data from a nearby computer system,and view it as if it were on their own terminal.These sets are readily availble if you know where to look,and furthermore,this form of eavesdropping is -not- illegal in the UK.This works by pulling in EMR from the target set,and decoding or emulating the info for human viewing.In the US,it is actually illegal to try and prevent this,but the UK has still to realise that it falls outside its current legislation on telecommunications.(TEMPEST is a Passive device for intercepting un-intentionally transmitted data). The MoD will no doubt be taking steps to minimise any emanations that could compromise its security,but until BOXER goes on-line next year,we wont really know.It -would- be intresting to see exactly what goes round that particular network... ---=[AZTECH]=---