2010-06-25

Opportunity Sol 505 (Spirit Sol 525)

Six fucking centimeters. That's it. Six fucking centimeters is all the distance we made yestersol.

It's my fault. In the drive sequence, we retained most of the limits that had been set for the previous drive. We were just passing over the ground they had meant to pass over, so we figured the same limits would be fine.

Frankly, this was just lazy. And it's what stopped us. As it happened, Opportunity's left bogie articulated just a little more than we expected it to as it climbed the ripple that's under us now (North Ripple), and that terminated the drive. After six centimeters.

We really, really should have caught this in advance. The bogie limit was 14 degrees, and if we'd checked the telemetry, we'd have noticed that it was already just over 11 degrees -- not a lot of headroom there. The really painful part is that I thought of that while we were doing the sequencing, and then got distracted by something and never got back to checking. Oy vey.[1]

And this changes our plans for the weekend. We'd originally intended to IDD the far track -- the one we were driving to -- but since we didn't make it there, that's out. So it'll be a touch-and-go sol, where we IDD whatever happens to be close to us and then try to drive the remaining 69cm or so to the far track.

Assuming we're capable of that much.

Both Jeff Favretto and Steve Squyres are philosophical about the drive result. "This is just a consequence of using conservative limits," Jeff says. And conservative limits are exactly what we promised the Mars Program Office we'd use, adds Steve. But as usual, things aren't that simple. Apparently, the Program management has been putting some pressure on John Callas (acting project manager while Jim Erickson is out of town). He's getting it from both sides: some people are complaining that we're being reckless cowboys; others, that we're too conservative. They want us to be successful without making any mistakes, which is fine to a point, but they've pushed it far enough that it amounts to demanding that we be lucky.[2] I'm glad I don't have his job.[3]

Squyres might have been getting some of that "too-conservative" flack as well. "I want you to drive safely," he tells me, "but I should say that if we don't make it to the target after this attempt, we're just going to IDD whatever we've got and then get out of here." "So, no pressure?" I ask. "Yeah," he grins, "no pressure."

The IDD work is fairly straightforward, and I get it more or less knocked off in a couple of hours. It initially looks like one of the targets the science team chose was too close to our right front wheel, but the simulation says it's OK, so I move on.

The bulk of the sol's work is in arguing over the drive. Not in implementing it, just in arguing about it. I go through every single parameter limit and check it against both the current state and the RSVP-simulated maximum, trying to pick sensible numbers for everything. We're left with a small set of numbers to argue about, the suspension limits -- most crucially, what should be the proper settings for the bogie limits that stopped us on 503? This turns into, I kid you not, a couple of hours' discussion at least.[4]

Maybe this is all ridiculous. The distance of this drive is about equal to a single wheel rotation -- how much trouble could we get into, even if we abandoned the limits altogether? But at this point we're so damn paranoid that we're ready to entertain the ridiculous.

The number we eventually pick for the max bogie limit is 25 degrees. That's about twice what we're at now, and just 5 degrees shy of the hardstop. It's almost impossible for us to actually hit it, and all things considered, I can feel OK about that. If we actually do reach that limit, something really strange will be happening, and we'll want the drive to stop. Which is what the limit is for.

Our paranoia doesn't end there. Steve asked me earlier in the day for a list of everything that could possibly stop the drive, and one thing that could stop it is for the IDD work to fail -- that would prevent the drive from even starting. So I look back at that, and the IDD target that concerned me before starts to look even more worrisome.

RSVP can show something called the "collision volumes," which RSVP visualizes as a set of purple boxes that are slightly bigger than the rover parts. They're like the rover is wearing bulky translucent purple padding all over. The on-board flight software uses these collision volumes to detect a pending self-collision; the idea is to be conservative about the possibility of collisions during (especially) IDD work -- if the padded version of the rover doesn't collide with itself, then the real thing won't. RSVP also tells us whether these collisions will occur in the sequence we've planned, and since the simulation is happy, we should expect that the real thing will be okay.

And yet ... when we're inspecting the nearby IDD target, the collision volumes come so close to intersecting that we have to really work to show that they're not. What if the flight vehicle is just a tiny bit different from this simulation? It shouldn't be, but as I said, we're pretty paranoid now, and the difference would have to be only a tiny one -- maybe small enough that it would have gone undetected until now -- to screw us up once again.

I'm just being paranoid. I'm sure I'm just being paranoid. But I'm not the only one, and we end up deciding to turn off those collision checks during the close approach. We're not coming so close that there's a risk of an actual collision, just a risk that the collision volumes -- the padding -- could collide, and turning them off means we can be sure it won't happen.

I won't feel good about any of this until Monday.

[Next post: sol 528 (Opportunity sol 507), June 28.]




Footnotes:

[1] This exact incident, if I recall aright, is why checking the suspension history from the previous drive is something I now do every sol. It's bad enough to make a dumb mistake, much worse to repeat it.

[2] Fortunately, this was the last time that Mars Program ever made such a stupid demand. cough

[3] Still true. And how.

[4] In a way, this is one of the things I really love about my job: the obsessive focus on getting these things right. We might spend an hour arguing about a degree, or a millimeter. Sometimes those discussions are horrendously painful when I'm in the middle of them, but when I take a step back, I love being in an environment where people are that passionate about even the little things.

No comments: