How to add points to multi-panel Lattice graphics

I'm making box-and-whisker plots in Lattice with bwplot. I want to add points for means. I've figured out how to do this for simple plots using panel.points. However, when I make a multi-panel bwplot using + and outer=TRUE, I don't know how to put different points in different panels. This example illustrates the problem with my attempted solution, which doesn't work. (For the sake of simplicity, it puts "x"s at arbitrary points rather than at means.)

# make some data:
df <- data.frame(var1=rnorm(20), var2=runif(20), cat=rep(c("Z"),20))
# cat isn't doing anything, but I couldn't get the mwe to work without it.

bwplot(var1 + var2 ~ cat, data=df, pch="|", coef=0, outer=TRUE, 
       panel=function(data, ...){
         panel.bwplot(...)
         panel.points(x=c(.5,-.5), pch="x", cex=2)
       }
      )

(outer=T combined with var1 + var2 is what causes two panels to be created. See below for a simpler example.)

What I get are two panels, with a box-and-whisker plot in each. "x"s are placed over each plot. In each panel, there is an "x" at .5 over the plot. Rather than using -.5 in the second panel, the process starts over--Lattice grabs the first point, .5, again.

How can I get different points displayed over plots in different panels?

Thanks.

Later addition: I realize now that this issue has nothing to do with + and outer=TRUE per se. The same behavior can be gotten using routine Lattice conditioning:

df <- data.frame(var1=rnorm(20), cat=rep("Z",20), cat2=rep(c("M","F"), 10))

bwplot(var1 ~ cat | cat2, data=df, pch="|", coef=0, outer=TRUE, 
           panel=function(data, ...){
             panel.bwplot(...)
             panel.points(x=c(.5,-.5), pch="x", cex=2)
           }
          )

You should use the accesor function panel.number() to choose the x value for each panel:

df <- data.frame(var1=rnorm(20), cat=rep("Z",20), cat2=rep(c("M","F"), 10))

bwplot(var1 ~ cat | cat2, data=df, pch="|", 
           panel=function(...){
             panel.bwplot(...)
             panel.points(x=c(.5,-.5)[panel.number()], 
                          pch="x", cex=2)
           }
          )

Edited: However, if you want to add points for means, you don't really need panel.number. You can define a panel.mean function (following the example by Deepayan) to compute and plot them:

panel.mean <- function(x, y, ...) {
    tmp <- tapply(y, factor(x), FUN = mean)
    panel.points(tmp, pch = 20, ...)
}

bwplot(var1 ~ cat | cat2, data=df,
       panel=function(...){
           panel.bwplot(..., pch='|')
           panel.mean(...)
           }
)