using two scale colour gradients on one ggplot

I would like to combine a colour scale gradient for the points on a scatter plot together with a colour scale gradient for some text that goes on the plot. I can do them separately as shown in my example below, but i can't seem to put them together...is there a way of doing this?

Here is my example code of the two types of plots (p and p1) that I want to combine

l <- data.frame(prev=rnorm(1266), 
            aft=rnorm(1266), 
            day=as.factor(wday(sample(c(2:6),1266,replace=TRUE),abbr=TRUE, label=TRUE)), 
            month=as.factor(month(Sys.Date()+months(sample(0:11,1266,replace=TRUE)),abbr=TRUE, label=TRUE)), 
            ind=c(1:1266))
cors <- ddply(l, c("month", "day"), summarise, cor = round(cor(prev, aft), 3))


# below the text gains the colour gradient
p <- ggplot(l, aes(x=prev, y=aft)) + 
    geom_point() + 
    scale_colour_gradient(low = "red", high="blue")+ 
    facet_grid(day~month, scales="free_x")+
    geom_text(data=cors,aes(label=paste("r= ",cor,sep=""), size=abs(cor), colour=cor), x=Inf, y=Inf, vjust=1, hjust=1, show_guide=FALSE)+
    geom_hline(aes(yintercept=0))+
    geom_smooth(method="loess")
p

# below the points gain the colour gradient
p1 <- ggplot(l, aes(x=prev, y=aft)) + 
    geom_point(aes(colour=ind)) + 
    scale_colour_gradient("gray")+ 
    facet_grid(day~month, scales="free_x")+
    geom_text(data=cors,aes(label=paste("r= ",cor,sep=""), size=abs(cor), colour=cor), x=Inf, y=Inf, vjust=1, hjust=1, show_guide=FALSE)+
    geom_hline(aes(yintercept=0))+
    opts(legend.position="none") +
    geom_smooth(method="loess")

p1

I do not expect that this can be done. A plot only has one scale per aesthetic. I believe that if you add multiple scale_color's, the second will overwrite the first. I think Hadley created this behavior on purpose, within a plot the mapping from data to a scale in the plot, e.g. color, is unique. This ensures that all color in the plot can be compared easily, because they share the same scale_color.