NASA is reporting that it has lost contact with another CubeSat that is part of the ten that were launched on the Artemis mission, CuSP. https://blogs.nasa.gov/sunspot/2022/12/08/artemis-i-payload-cusp-cubesat-mission-update/

While we do not definitively know what caused CuSP to fail (and the CuSP team may regain control of CuSP), there is some data re anomalous battery operation that perhaps gives us some insight and can be used as a teaching moment. "An unexplained battery anomaly also occurred at the end of the initial data collection period. Two minutes prior to the end of the data collection period, one of the battery cells suddenly experienced a temperature spike – jumping from 34 degrees Celsius to more than 168 degrees Celsius in under a minute. The temperature of the anomalous cell subsequently increased from approximately 34 degrees Celsius to about 80 degrees Celsius before loss of contact"

Most CubeSats today use Lithium-based rechargeable batteries, primarily because of their high energy density, and because Lithium batteries are so prevalent in mass-produced consumer products like laptops, smartphones, power tools and cars. Rechargeable (also called secondary) Lithium cells are available in many different chemistries, and one can generalize that most commonly-used Lithium cells like to operate in a relatively narrow temperature range of roughly 0C to +40C. Most cells can discharge (albeit with lower performance) at temperatures below freezing, down to -20C or maybe a bit colder. Similarly, some chemistries work even more efficiently at temperatures above +40C to +50C, but major manufacturers like Panasonic and LG generally recommend not charging above +45C or so. Most CubeSat batteries are composed of multiple cells, in series and in parallel, to achieve the desired voltage range and current delivering capability, and stored energy; 2S2P (an "8.4V, 50Wh battery") and 4S2P (a "16.8V, 100Wh battery") are common cell configurations.

The report from CuSP that a single cell's temperature rapidly climbed from +34C to +168C tells us a couple of things. First, +34C is a reasonable operating temperature for a cell. CuSP's internal temperature might have been around +34C and very little current was flowing through the battery. Or perhaps CuSP was at a colder temperature but the battery was being charged --  charging inherently causes cells in the battery to heat up, with higher charge currents resulting in more cell self-heating. Either way, +34C is a perfectly reasonable operating temperature for Lithium-based cells; +168C is not. It's not clear what type and size of Lithium cells CuSP used. But if the +168C reading is correct, and if the cell in question is a typical 18650 (cylindrical Li-Ion) cell, in rough numbers it took around 5,000 Joules (or 5,000 Watt-seconds) to raise the cell's temperature by 134C. Given that the change in temperature happened "in under a minute,"  then that reflects a hefty dose of energy that is in excess of the roughly 50-60W of power that comprise CuSP's solar array, applied over that minute. Therefore, it seems unlikely that CuSP's EPS failed to the point where it was attempting to overcharge the cell, and suggests that another source of energy is being responsible for the cell's temperature rise. That is probably a failure in the cell itself. This kind of failure (typically an internal short circuit) may be localized to the cell itself if each cell is individually fused. If not, then connected cells may also dump their charge into the failed cell, heating it up further. When a runaway event like this happens, one can generally assume that the affected cell is no longer functional. At a minimum, the individual cell no longer provides any energy to the battery (a pack of interconnected cells), and it may or may not inhibit current flowing through any series-connected cells elsewhere in the battery.

When a cell failure like this happens, several issues rise to the fore: First, the battery voltage will have fallen by one Lithium cell's worth (typically 4.2V when fully charged). In, say, a 4-cell battery with all the cells connected in series, the battery is now operating at 3/4 of its normal operating voltage. In a  16.8V Li-Ion battery, that means that the maximum voltage achievable is now just 12.6V, which is very close to the undervoltage limit of the three remaining cells. At best, you might end up with a battery that now has an operating voltage of 12V to 12.6V, which is a tiny fraction of the total energy of a 4S, 16.8V battery. This narrow operating range would be difficult for most EPS and battery systems to handle correctly. Second, the battery failure may take down the entire power system, depending on how the EPS is architected. Some EPSes can operate without batteries, so a working ADCS system could conceivably keep CuSP alive if there are no loads on the system that exceed the solar array's power output. Conversely, a battery failure could even physically harm the EPS, ending the mission. Lastly, if the battery has cells connected in parallel, and one cell has failed, then the current through the remaining cells will be limited by the remaining parallelism; i.e., a single cell's failure (with automatic isolation via a blown fuse) in a 4S2P battery yields a 3S2P + 1S1P battery, which is effectively a 4S1P battery from the standpoint of how much current can be drawn from the cells. For most EPSes, this will also lead to difficulty in charge balancing, end-of-charge threshold issues, etc.

In summary, from the contents of the NASA blog, it appears that there was a failure internal to a battery cell inside CuSP, and this may have taken CuSP's EPS down with it. If NASA is able to establish communications with CuSP, then that may suggest that CuSP's battery is no longer functional, but that the EPS is able to operate without it.

Pumpkin's approach to power systems has been quite different from the usual CubeSat offerings. We've applied automotive-grade multi-level safety, redundancy and intelligence to our BM2 intelligent battery module (a 4S2P Lithium-Ion battery), and our EPSM1 is completely capable of operating with two, one or zero active BM2s attached to it. Morehead State's Lunar Ice Cube (one of the Artemis 1 CubeSats) has been successfully commissioned on-orbit, with Pumpkin's EPSM1 and BM2s aboard.

While it is not unheard of for Lithium-based cells to fail internally and thereby cause problems that ripple through spacecraft power systems, Pumpkin's EPSM1 and BM2 are designed to give the host CubeSat a fighting chance at continued operation in these conditions.

Morehead State's Lunar IceCube 6U CubeSat is on its way to the Moon! According to NASA GSFC, Lunar IceCube made contact with NASA's Deep Space Network's Goldstone 24 dish after deployment as part of the Artemis mission.

Lunar IceCube utilizes a Pumpkin two-wing, 126-cell solar array with each wing individually articulated via Pumpkin's DASA, a kind of Solar Array Drive Assembly (SADA) that incorporates a rad-hard H-bridge designed by GSFC. Solar array power feeds into Pumpkin's EPSM1 power system, to which two Pumpkin BM2 batteries are connected to give a total of 200Wh of reserve power. MSU designed a custom tabbed-style 6U CubeSat chassis as the structure.

The EPSM1 was chosen for this mission because of its ability to handle the difficult load that the on-board cryocooler for the BIRCHES instrument presents, and because Lunar IceCube needs a lot of power to operate its thruster as it transits from Earth orbit to a lunar orbit. Lunar IceCube is in group "A" below:

Of note is the fact that Lunar IceCube and several other CubeSats were integrated onto the Artemis LV in the fall of 2021; Lunar IceCube was not afforded the opportunity to top off its batteries a year later.  When inhibited by their separation switches, the Li-Ion cells in the Pumpkin BM2 batteries experience the same self-discharge rate as bare 18650 cells manufactured with Li-Ion cell chemistry (typically around 1.5% per month). Despite the 15-month storage time and being outside for several weeks prior to the Artemis 1 launch, the all-Pumpkin power system weathered the conditions just fine, and Lunar IceCube's initial commissioning appears to have gone smoothly. Additional BM2 features like automatic cell balancing and heaters will come into play over the lifetime of the Lunar IceCube mission.

From its current orbital position around Earth -- where it is subjected to considerable radiation -- Lunar IceCube will take another six months to reach the moon and begin its lunar capture maneuvers.

Pumpkin's EPSM1 with BM2s is the class-leading power system for small satellites, and has been delivered to multiple customers for upcoming missions.

You can follow Lunar IceCube's progress on its way to the Moon online.

Because of a third-party delay, one of Pumpkin's 12U buses has undergone ground-based testing for several months longer than originally planned. This has presented the opportunity to fine-tune Pumpkin's GUTS flight software for enhanced performance. Because this 12U program only purchased one radio (the Flight Model), Pumpkin's software team developed a suite of tools to ensure reliable radio communications without having 24x7 access to the flight  hardware.

As part of our testing, a ground-based Engineering Model is used to first validate GUTS, and then new code releases code make their way to the Flight Model. One of the critical GUTS components is the radio service that implements the CCSDS protocol over a commercial S/X-band radio.  In order to test the radio service without access to the S/X-band radio, Pumpkin wrote a radio simulator. In the real-time performance plot below, the radio service (running on a Pumpkin 1GHz BeagleBone Black (BBB) SBC in a Pumpkin RS3) is communicating with the radio simulator (on a Xeon-based desktop PC) over a 100Mbps Ethernet link.

By using the simulator to develop and fine-tune the simulator, GUTS's CCSDS radio service can now deliver continuous throughput as high as 45Mbps (limited by the BBB SBC). While the actual radio may limit on-orbit performance, this test setup demonstrates the current upper bound of GUTS' radio service's performance in the existing 12U SUPERNOVA architecture. Further optimizations may raise the radio service's performance even further.

At the end of the cold-temperature BM2 battery testing mentioned in the previous blog post, the BM2 was allowed to rest. This simply means that the overall power system was not drawing any power from the BM2, and the BM2 was fully charged to its terminal voltage (16.2V in this particular test). The resulting cell voltages illustrate the efficacy of the cell balancing algorithm -- there's a maximum spread across the four cells of 5-10mV.

The BM2 intelligent battery module includes an active battery balancer that automatically seeks to equalize the voltages of all two (2S4P) or four (4S2P) cell groups in the BM2's total of eight 18650-size Lithium-Ion cells. The balancer is active when the battery is charging and when the charge current has dropped below a prescribed limit.

Most Li-Ion batteries cannot be charged at temperatures below 0ºC, and the BM2's LG MJ1 cells are no exception. The BM2 cell heaters for this cell chemistry automatically enables the heater at 5ºC as the temperature drops and disables it once the temperature rises to 7ºC.

As part of low-temperature testing being performed at Pumpkin, a neat telemetry capture came up this evening. The thermal chamber is set to -5ºC, the BM2's heater was in the default AUTO mode, and the charging current into the BM2 was limited to 1A to simulate a wimpy charger. The cell heaters consume around 8W; battery energy is 100Wh.

As the heaters turn on and off,  the current flowing into the cells decreases and increases, respectively (because charge + heater current is limited to 1A). Each time the charge current increases, the cell voltage increase with it, but individually. The telemetry capture shows the four individual cell voltages as the BM2 approaches its final pack voltage. You can see the active cell balancing in cell 2 (the purple trace): it started out around 20mV higher than cell 4 (the yellow trace), but within 45 minutes of all of the cells charging, these two cell voltages have converged. All four cell voltages will converge further as the BM2 continues operating like this. The cell balancing is slow but effective; we typically see mismatches of under 5mV across four cells, when the balancing algorithm is allowed to run its course.

The cell balancing, cell heaters and real-time telemetry (and nearly 100 other data points) are available to all BM2 users as standard features of each BM2.

After over 13 years of holding prices constant, inflation and the global supply chain (GSC) crisis have finally forced Pumpkin's hand. Chips that were easily available in 2019 for $4 are now costing $65, if you can find them. So it was with great reluctance that we raised our prices today.

We will honor the older pricing until May 31 for those customers who have received quotes or contacted us with pricing between Jan. 1 and April 8. For all other customers, the new pricing takes effect immediately.

Did you know that Pumpkin also develops and sells a unique RTOS that is targeted for MCUs with small memory footprints? All of Pumpkin Space's microcontroller-based modules utilize a Microchip PIC24E MCU, with Salvo as the foundational RTOS. In these applications, Salvo enables event-driven, priority-based tasks to run on the MCU in a highly decoupled manner.

Pumpkin is not the only Salvo RTOS user -- today we received an email from a company whose products you are likely to see when visiting a large world-wide fast food chain:

Salvo is still great, in no uncertain terms, for our application on dsPIC it's been a God-send. We build 40,000 dsPIC33
boards a year, and at present 35,000 ship with Salvo running on them. Harold F.

While there is a learning curve associated with learning an RTOS, once learned, the benefits of the structure that the RTOS brings to an application are substantial. And a carefully-crafted RTOS like Salvo can easily be configured to have  zero impact (yes, zero!) on the real-time responsiveness of the application.

In 2009-2011, Pumpkin developed and delivered a series of twelve 3U-sized CubeSats to the National Reconnaissance Office (NRO) -- these were the Colony I buses. These 3U CubeSat-size buses had fixed and deployable panels, a full 3-axis ADACS with coarse sun sensors and a magnetometer, an 8051-based C&DH processor, multiple batteries and over 1000cc of payload volume. The NRO distributed Colony I buses to various organizations ranging from the Naval Research Lab (NRL, with their QbXI & QbXII CubeSats) to USC's ISI, with its Aeneas CubeSat. 

Sometime in the not-too-distant future, one of thousands of Pumpkin SUPERNOVAs is likely to fall back to Earth. Some time will pass, and someone will find it, in a field, resting, its payload long gone, and yet seemingly unscathed due to its robust construction.

Almost unscathed -- if you look closely, one of the SAP cover screws (made from 18-8 SST) burned up on rentry, and the -ZA1 SAP cover has worked itself loose ...