The Solectek Saga August 21st, 1997 School District 11 of central Colorado Springs is a rather typical urban school district. With over 55 school buildings, over 30,000 students, its District Headquarters is near the center of the school locations, with the furthest school being 7 miles from the District location - where all the networking lines converge. But the District Headquarters is, geographically speaking, 'down in a hole' - the lowest point on Unitah Street in central Colorado Springs surrounded by gently rolling terrain, and by 120 foot tall, mature, trees. Therein lies the tale of this effort to link some schools with no-licence spread spectrum radios at least at the T-1 level of connectivity speed, as part of the National Science Foundation (NSF) Wireless Field Tests for Education Project for which I am the Principal Investigator. I selected District 11 as representative of a central school system in a medium sized (300,000) American City, which was only beginning to develop (1995) complete school-to-school and Internet connectivity to classrooms. Only in 1996 did the voters approve a large bond issue which would buy full technology for all schools, and connectivity. BILLY MITCHELL WOULD BE PROUD One 1,500 student high school, Mitchell, named after the Aviation Pioneer, with a substantial inner city minority population had been equipped at the District's expense with a 'demonstration' computer lab, off its school Library, equipped with over 40 Mac Power PCs on an internal LAN. Thus, because there would be a significant load on the wireless link to the District Headquarters and the distance between the school and the District Headquarters where the outside Internet links from MCI and the network firewalls were being placed, was a nominal 3 miles (the furthest school building was 7 miles), I chose the Mitchell to District link as the basis for the NSF test. An agreement with Kenneth Burnley the Superintendent and the then Principal of Mitchell was executed for the NSF project to try and link up that school to the Internet POP at the District Headquarters wirelessly at high enough speed to serve the needs of the school. I would work with Becky Modic, the Computer Lab coordinator at Mitchell. SOLCTEK'S AIRLAN I then contacted Solectek of San Diego as the manufacturer of a late model (1995) spread spectrum no-licence wireless radio to try out. Solectek chose to work through its regional VAR, LanTech, to support the project, and agreed to the loan of a pair of AIRLAN/Router 200E radios, until the tests were over, after which time either our NSF project would buy the radios and give them to the School District, leaving them in place as operational links to the Internet if they were useful to them, or ship them back to Solectek. LanTech also agreed they would do the installation and configuration work just as if the school had ordered and paid them as vendors. We could then observe, not only the technical work and results, but all the coordination that had to be done between wireless vendors and school systems unfamiliar with the needs of radios or of the schools. The specifications on the AIRLANs, with a retail price of $6,000 each at the time were, as described by Solectek: *Solectek Wireless AIRLAN Technology Transmits data up to 25 miles under ideal conditions - 16* miles in typical rural environments - 7 miles in typical urban environments Data Rate: 2 Mbps Spread Spectrum radio technology: 2.4 GHz No FCC license required for operation IEEE protocols: 802.3 Ethernet (Router 200E) IEEE protocols: 802.5 Token Ring (Router 200T) Wireless media access protocol: CSMA/CA Diagnostic LED indicators: transmit, transverter unit detect, DC power OK, synthesizer lock Power output at antenna: 4 watts EIRP** - the maximum allowed by the FCC Adjustable power output and receiver sensitivity Available with DES encryption Router 200E supports 10BaseT (RJ-45) and 10Base2 (BNC) and for external transceivers (AUI); thick Ethernet (10Base5), plus fiber optic connectors Router 200T supports Type 1 shielded twisted pair (DB-9) or Type 3 unshielded twisted pair (RJ-45) One year limited warranty with extended warranty available *Requires line of sight and a clearance of 95 feet for antenna pathway (Fresnel zone) **When configured with High Gain Directional Antenna Software: Codeveloped with Cisco Systems (tm) Transparent MAC layer bridging for compatibility with all major network operating systems (NOS) including Novell NetWare, Microsoft NT and Windows 95, HP LAN Manager, Banyan Vines, Unix, SNA (Router 200T), IBM LAN Server (Router 200T), OS2 (Router 200T), and others IP and IPX routing IP subnetting Concurrent routing and bridging Simple Network Management Protocol (SNMP) MIB II compliant IEEE 802.1D Spanning Tree Protocol (Router 200T) Remote router management Antenna alignment utility Custom filtering Internal unit: Height: 3.75 in. Width: 14.00 in. Length: 15.50 in. Weight: 13 lbs. Power Consumption: 4 A max @ 115-230 VAC Certification: Power supply UL, CSA, TUV / Bridge FCC A / RF FCC C Temperature: 50: F to 104: F Long Range Transverter (mast mounted): Height: 1.0 in. Width: 5.5 in. Length: 8.0 in. Weight: 2.85 lbs. Temperature: -22: F to 158: F Solectek Corporation 6370 Nancy Ridge Drive, Suite 109 San Diego, CA 92121-3212 RADIO REALITY TIME LanTech engineer Dave Walsh made the first site survey for deployment of the radios in March of 1996. He learned that there was not direct line of sight between Mitchell and the District Headquarters, where all the wired data lines terminated, so it would take at least 3 radios, the middle one being a relay radio somewhere. One possibility we had pointed out was a mast antenna at Lunar City Park, roughly half way between Michell and District, which had a relatively clean look at the District Headquarters, but was pretty badly masked by buildings and trees in the direction of Mitchell to the east. A detailed analysis of this possibility ended up with it being ruled out. He looked at distant Austin Bluffs, which were visible to both Mitchell and the District (still with 120 foot trees in the way). But power and permission to erect a tower on private property might be obstacles. That site would be a last resort. The 3 mile link was not going to be easy. (But then if it was, we would not have learned much about the limits of wireless solutions between schools in a typical city.) By this time Walsh called upon the radio engineering staff of Solectek, who consulted their Geographic Data base, and consultations began. From maps of the District, it was apparent that some District schools lay across the lower lying center of Colorado Springs on the foothills to the Rockies, to the west. The site survey and analysis began to focus on Coronado High School high on a mesa in western Colorado Springs. With the idea of shooting the wireless beam with directional antennas 9 miles across from the roof of Mitchell to the roof of Coronado, and then back down into the 'hole' which was better seen from from Coronado. The first idea was that Coronado would become a radio 'relay' point only, with the Airlan radio operating by itself in a room on or near the roof, and the outside antenna receiving from Mitchell, and resending to the District. Such a one-radio relay would cost the link half its speed, as the center radio would have to work serving two distant radios. CRITICAL SITE SURVEYS By this time, mid to late 1996, several site visits from the chief Solectek Radio Engineer, Dave Walsh and LanTech, and myself had been made. We (NSF Project) stayed out of the way of LanTech, only observing, and representing the project to the school administrators and technical staff, while LanTech and Solectek worked to find a good solution that was reliable in the long run. The last thing either Solectek or LanTech wanted to happen was to have a lower-functioning link, or unreliable service to the school, giving, if not that product, then the idea of wireless, a bad name. There are many cases of schools and colleges just 'buying radios' and putting them up, or having a hardware reseller - instead of well qualified VAR with radio engineering expertise - put them up. But the necessary trips also underscored the importance - and potential cost (professional technical services) of the Site Survey before installing anything. This actually took over a year to get to the final installation design. Partly because it also ended up involving changes in the LAN and IP infrasctructure of the District. I could easily estimate that, if paid for up front and seperately from the cost of the radio, antenna, mast, and connecting equipment and installation, this sight survey took at least 15 man-days, and would have cost somebody $7,500 - more than the markup of the radios could bear. This aspect of deployment of wireless - a Site Survey -is, in our NSF project experience, the most overlooked - by school officials, yet vital, aspect of wireless use. At least one national company that works with schools in the east and south, charges $800 a day for site survey work. It cannot be overlooked or wished away, or there is a risk that, after having purchased radios, technical problems and solutions not observed during cursory, or just 'visual' surveys, that the radios will not perform well, or additional costs are incurred by re-engineering the wireless architecture. Solectek employed on their visits, a sophisticated Spectrum Analyzer which permits the proposed 'path' to be analyzed for effective throughput, link margin, and potential sources of interference. Because of the terrrain, the radio sites turned out to be difficult - not obvious at the beginning. So the time and cost spent in this professional analysis could have doubled the cost of the radios alone in this unique case. LANTECH'S RECOMMENDATION Since the Mitchell and Coronado radio sites were 9 miles apart, it required the 'Long Range Transverter' dish antenna - about 2 feet across, solid, mast mounted, be used as a directional antenna. At 2.4Ghz the 'fan' of the antenna beam is only about 15 degrees of spread. Walsh was hoping that, when the Coronado Radio antenna was aimed back at Mitchell from the roof, that the arc of coverage would also cover, reliably, the location of the District Headquarters in between, slightly off the direct line path between Mitchell and Coronado. But tests, using the built-in diagnostic system that comes with the Airlan radios (requiring only a PC and Monitor to be attached during testing and installation) showed that the arc was too narrow to deliver and receive signal from both sites to the east in a robust manner. So Walsh finally announced to me that, to make the link, with Solcteck AIRLANS between Mitchell and the District work at least at T-1 half duplex speed, and reliably, that it would take 4 Airlan radios, two at Coranado ethernetted together, one roof and mast mounted antenna aimed at Mitchell directly, the other at the District Headquarters. That the mast at Coronado had to be at least 100 feet off the ground level (about 15 feet of mast on the roof), the mast on the roof at Mitchell at least 30 feet high, and a 120 foot mast would have to be erected at the District Headquarters, in order to clear those massive trees and get down 'into the hole' which was, however the point at which all school data lines converged. The total costs, then, for the T-1 link would have been at least $24,000 for four radios, and about $8,000 in masts, or $32,000. MY DECISION I said no to the 4th radio and final link to the District. That we had learned what we set out to learn. That in the particular case of Mitchell and School District 11, that trying to tie all the schools into the District Headquarters 'down in the hole' by wireless was a too costly proposition. In this respect the project differed from the installations made at School District 20 (Air Academy) and reported on my me - in that the District Headquarters sat on a hilltop, which made the best links to and from the center, with more problems at the periphery. Now we knew that. An important thing to know by the school's technical staff. That the time it would take to amortize the competitive costs of US West T-1, which was being discounted to the school system, averaging $250 a month, would take too long to justify the wireless on the grounds of cost savings for a 4 radio link for just one school. However, having said that, the shape of a wireless layout better suited to School District 11 began to emerge. BETTER SCHOOL USE OF WIRELESS During these deliberations, and consultations with the new District network engineer, we learned that the high schools had, by this time, dedicated wired T-1s for data. That, for the purposes of the test and demonstration of capabilities, we could just link Mitchell and Coronado High Schools by wireless T-1, then sharing Mitchell's load with Coronado's on the one wired T-1 between Coronado and the District. That the loads were not that heavy. But also, that the District was organized on a 'feeder schools' system. That for each high school in the city in a geographical sector, both middle and elementary schools were associated with it, and generally were in closer proximity to the high school than any other schools. This indicated to me that the probable best 'total' connectivity solution for School District 11, was to extend US West T-1 wired lines from the District to the High Schools, but consider 'sharing' those lines - saving substantial line costs, by extending data links from the high schools to their associated middle and elementary schools. I communicated that idea to the Superintendent and his staff, who took it under advisement. So the extensive analysis of the one link resulted in a broader strategic view of the integration of wireless with wired under the particular geographical layout of this district. I decided to change the operational test, by getting an Airlan link just between Mitchell and Coronado, and 'share' the Coronado to District wired T-1 link. I got the permission from the respective Principals of the schools. So we did that. Our NSF project paid for Spectracom Company to come in and erect a 30 foot mast on Mitchell, a 15 foot one on Coronado, while LanTech installed Airlan radios in a roof room in the Coronado case, and the patch-panel room in the case of Mitchell. The District techs cut the wired telco T-1 data link between Mitchell and the District, and made ajustments in the routing tables. Except that it turned out not to be that easy, for the flat TCP/IP school district network, with an internal (not Internic assigned) IP addressing scheme, with the radios acting only as LAN bridges, not routers, it forced the district to assign all the IP numbers at Mitchell and Coronado to be the same set. The network architecture in District 11 was not well, or optimally, designed - it just grew like topsy. And in fact, at one previous point when it was not working properly with telco wired links to all 55 schools, the District had to hire outside engineers to trouble shoot it. Lacking a fully qualified network engineer on the district staff, exactly what the outside firm had done was not clear. So it took time, with some trial and error, to get the new network design to work between Mitchell, Coronado, and the District - relegating the Airlan radios only to be LAN bridges, rather than routers. It would have taken a costly and lengthy redesign of the entire school district network system to make it operate the way one that large should be laid out. That is one reason the District hired a new network engineer who came on board in the spring of 1997 just about the time we were installing the one radio link. The incremental growth of the school use of telecom and IP addressing, and routing was catching up with them. This is not an un-typical situation with American school districts, many of whom have been backing into technology, LAN networks, then Internet works over years of time, and out of current budgets. Seldom, because of personnel costs, do school districts have network engineers hired before the architecture is decided by a series of incremental decisions and installations. Fortunately for District 11, the Bond Issue with funds for technology, is permitting the District to get it right for the long run, although the potential recurring costs to the District for a completely telephone company wired solution linking over 60 total buildings across the city, has been a powerful incentive to take a hard look at the wireless, no-recurring-cost, solutions. Given the direction the District was going before the wireless option was introduced to them, I calculate that the annual data line costs would have reached $15,000 a month or over $180,000 a year payable to US West. A lot of advanced radios, requiring only a one time cost can be purchased with such funds. So it took more visits by LanTech, and its new well technically qualified marketing manager, Mike McDonald, working with the district techs to make the necessary routing modifications and IP assignments to get the link working properly. But, by Spring Break in 1997, the link began operating. One day the Computer Lab at Mitchell was traversing the Internet via a US West T-1, recurring monthly cost line (which was being shared at the school end between the Adminstrative VAX circuits, and the TCP/IP classroom data circuits) to the District, and the next, it was getting there via a spread spectrum wireless link that appeared to be operating at about 1Mbps. At the time we were deploying the radios, the district had two wired T-1s to every high school - one for voice phones, one for data. It had one T-1, split and channelized voice and data, to every Middle School. And it had a 56kbps dedicated telco link to every Elementary School. A flat network - which would be very costly in telephone company charges in the long run. At least $15,000 a month if the District used only telephone company circuits. The wired T-1 that had linked Mitchell to the District was actually a channelized, and shared circuit - part of it being used by the School for administrative purposes linked to a school VAX, not running TCP/IP, while the remainder was for the Computer Lab and future classroom computers. When all the changes were made in the network between Mitchell, Coronado, and the District, the masts put up, the radios installed and optimized, Becky Modic, who supervises the use of the large Mac Power PC lab reported that the wireless link to the net seemed to perform faster and more reliably than their previous wired link to the District. LARGER LESSONS LEARNED The timing of the installation of the NSF project Solectek radios was fortuituous. Just as the District was facing the costly extension of all 55 of those telco-based data links, with undoubted higher bandwidth needed in the future, our project, Solectek, and LanTech came along, and essentially educated the school's administration on the value and economics of wireless as an alternative to telephone company wired. (And in my own conversations with Kenneth Burnley, Superintendent during all this, I did not hesitate to point him toward the all-wireless neighboring School District 20 (Air Academy), where the 10 year savings would amount to $2.3 million over a wired school district. Only one period saw the network fail for several days. A record snowfall hit during a spring Colorado storm, and the heavy wet snow building up on one dish antenna at Coronado, bent it down 15 degrees, throwing the beam off line and halting the net. Until Dave Walsh realigned the antennas, the net was down. This was a rather exceptional event, since it was record depth of very wet snow. It did, however point up the potential value of 'hoods' on the exposed antennas in northern climes. THE WIRELESS HAWTHORNE EFFECT In the 1920s researchers observed that factory workers, when studied for their productivity, raised that productivity as a human reaction to being 'studied' and paid attention to. It was called the 'Hawthorne' effect. In many of our NSF projects, there seems to be a Wireless Hawthorne effect - where studying just one wireless link in a large organization, sensitizes and motivated the whole organization to delve into wireless solutions. In this case, the reality and events surrounding the putting up of just on wireless link in a system with 55 schools, has gotton the district thinking wireless and talking to wireless vendors - including LanTech. So it appears that, as the District starts to spend the Bond money for technology, it was incorporate wireless that it never even considered in its original Technology Plan, into the school networks, and thereby save the school and taxpayer costs in the long run. So, quite apart from the technical performance and comparative economics of the pair of Airlan radios installed in Colorado Spring's largest school district, the project can be deemed a success - for another large public school system facing large telephone circuit costs has found an alternative way to link its schools, classrooms, students and teachers to the Internet and each other. The cost of doing so may be more in the expertise required than in the plain cost of radio hardware. It was a lengthy, and therefore costly learning curve. But a 'technology transfer' from the rarified world of spread spectrum no-licence data radios, to a public school district has been accomplished. And the District wants to keep the radio link. So the NSF project is buying the radios and granting them to District 11. Everyone learned what we set out to learn. Dave Hughes Principal Investigator NSF Wireless Field Tests for Education dave@oldcolo.com http://wireless.oldcolo.com