Beulah Water Works District

Beulah Water Works District is a small surface water system west of Pueblo, Colorado with ~180 taps.  The original water plant was constructed in the 1950's and has been upgraded and added to in a variety of ways since. The plant currently runs a dual Neptune-type rapid sand filter and treats water from a mountain river a few miles away. The conditions of the raw water change seasonally and this requires regular changes to the backwash process made on the part of the operator.  In the early 2000's a company was brought in to automate the plant process with an Opto 22 PLC and update the relay system. There was one caveat, the old-style relay automation was to kept and work in tandem with the PLC. This is quite unusual but the water board at the time had their reservations regarding the reliability of computers. The automation company completed the relay logic but was unable to complete the PLC automation due to a major illness.  The PLC sat for years simply showing tank levels on a computer interface and nothing more. Operators had to manually engage the backwash process and had little control over the process. 

In 2014 Stanford Engineering was brought in to update the Opto 22 LCSX PLC to a SNAP PAC controller. We were asked to automate the tank level control system and pumps, add a text and email alarm system, and tie the controller to the relay logic governing backwash to automatically start backwashes based upon output from vacuum gauges. They also wanted the PAC to monitor which filter was actively being backwashed. Upon arrival we found the main relay panel to be messy, the relay panel on the filter itself was badly corroded due to lack of any gas sealing. (See pictures below) To make matters worse previous operators were handy with circuits and had run their own control system extensions to various places in the plant. There were no schematics or documentation except a basic ladder diagram of the relay logic on paper. Just another day at the office. 

We managed to tie inputs on the Opto 22 controller to a couple relays in the master panel to tell the computer what stage the backwash was in. To automatically begin the relay controlled process we tied the controller outputs to the filter select and backwash start buttons on the operator panel. The vacuum switches, tank level sensors, effluent pump relays, turbidimeters, chlorine analyzer and waste valves were tied directly to the controller. This allowed the controller to kill pumps on high NTU or low chlorine alarms and send an alarm. The vacuum switches governed the start signal for backwash and the waste valves were placed on an adjustable timer in the operator interface for filter flushing at the end of a cleaning cycle. 

Upon completion of this project in the summer of 2014 we brought suggestions to the water board for future system upgrades.  One of those suggestions was abandonment of the relay logic entirely.  Other suggestions included panel cleanup and rebuilds, numbering and diagramming of all wiring and changing the placement of the tank level sensor.  The board took this under advisement yet wished to stick with the relay logic as a backup to a PAC. In mid-2016 the relay logic system began to fail due to relay age, dirty power from the power company and corrosion. During this time the Opto 22 PAC controller worked reliably with regular upkeep and some bug fixes.  It was decided that the control system should be reworked and made into a purely digital system. 

Our Design

During our system design we had two lead priorities. Fully digitize the plant by tying every object into the PAC controller, and cut cost anywhere possible. We carefully went through every panel in the plant and created a prioritized list of needs. We then looked for items that could be reused, such as boxes, relays, panel lights and switches. The main relay panel was an absolute mess, one board member called it a "rat's nest". This panel needed to be mapped labeled and cleaned. An extension rack would need to be added to the master rack to control the backwash process. (The main rack was full) Unfortunately we didn't have enough hardware clearance to place it in the main relay panel, but a box that was holding a small backup UPS could be removed and reused to hold the extension rack. 

We determined that a large backup power system would be required for this system and it needed an active line conditioner and surge suppression to correct the daily fluctuations in the local power system. These issues caused problems in the plant a few times per week. We scoured various suppliers and found a UPS normally used in network data centers to be a perfect item for the job. This UPS powers the PAC controllers, tank level sensors, internet modem and firewall, and control relays for about 45 minutes. The UPS also has an output to the master control rack that indicates power status. This allows the controller to see that a failure has occurred and provides enough power to send an alarm to the operators and kill the necessary control outputs to protect the system from restarting in a bad state.  Since the plant currently has no backup generator this was essential.  Our aim was to increase reliability, decrease maintenance cost, and give the operator as much control as possible over the fine details of plant control. For example backwash and scrub pump timing.  The main tank level sensor was located on the effluent pipe going to the water tanks, which causes woefully inaccurate readings. We specified a custom built submersible sensor that was calibrated to the tank and our control system, which also contained surge suppression in the event of a nearby lightning strike.     

Finally, we kept in mind that our software would need to be compatible with the combined project planned for the neighboring Pine Drive water system. This software would take over automated control of the Pine Drive system in the event a flood or fire destroyed the Pine Drive facility. This project is slated for late 2018 and you can read more about it here.

The Results

As of this writing (February 2018) this system has been fully online for a couple weeks. The main operator panel was reworked and allows the operator to operate backwash, alternate filters, see backwash status and alarm status without a computer interface if needed. They can also see readouts of chlorine, turbidity and tank levels on the panel. these items were reused from the old system. 

 The computer interface for the operator was updated to show longer trend graphs and shows the backwash process in its entirety. Operators can adjust backwash and surface scrub times, flush the settling basin, modify filter to waste time, set tank levels, see system alarms, test pumps and valves and more.  As always, we kept the feel of the interface simple and not too overwhelming while still allowing full control. 

Though the project was slated to take two weeks it took three to complete, but we did remain within budget. The lengthened schedule was due to the fact that every day we walked away from the job site the plant had to be fully operational due to winter leaks that had emerged on the system. Each day operators were briefed regarding progress and how to operate certain portions of the plant that were being worked on if the need arose. This process went smoothly without any major issues. To speed up the reworking process electricians were brought in for 7 days to run new conduit, replace and move panels, install surge protection and trace circuits. Each circuit had to be individually traced then either be rewired or removed. The main panel (the Spaghetti Monster) proved to be a unique challenge that proved tedious and time consuming. Each wire was numbered and recorded on a wiring schedule for future reference. During the project the board also decided to upgrade the plant lighting to LED for better light and energy use reduction.  We removed an old UPS in a relay box in the process room. Having discovered the UPS was only being used as a surge protector for one item.  We downsized the relay box and placed a small line protector on the outlet going to the item instead.

During our electrical work grounding issues in the original wiring were identified that will need to be addressed in the near future. This problem has an effect on some sensor readings in the plant due to electrical noise from poor grounds. For now, our software has been modified to compensate for the problem.   

Operators enjoy the greater flexibility of the plant and have been able to tweak the process to get more water through the filter before it must be backwashed. Failures due to dirty power have gone to zero and the water board members are pleased with the results. This system is designed to be remotely accessed securely from our offices. We are able to connect and check system health, walk operators through issues, perform software upgrades and modify the control software without a site visit. This allows us to cut cost for system maintenance to a minimum. The system was designed with future needs in mind as well. Room was left on the control racks and in conduit and panels for future additions to the process. Those additions will now require much less time and money to install. We expect this system to last the people of Beulah Valley a good many years and will check in with updates as we collect more operational data.