How to properly use the MouseMotionListener to pre

I have made a small Conway Game of Life program. I'm about 80% done. I have used a grid of JButtons as the cells. Right now I have a ButtonListener on every button so you have to one-by-one draw the pattern you want by clicking on individual buttons. I want to be able to click and drag the mouse and select buttons quickly. I used the MotionListener class, implementing the MouseMotionListener and coded the mouseDragged method in an identical fashion as my actionPerformed method in my ButtonListener class.

I thought the logic should be the same but I'm definitely missing something. I played around with it a bit, thinking that it was just selecting and de-selecting over and over faster than I could tell. I added a check to make sure that it did not try and change the same button back-to-back but that did not help. Here is my MotionListener class:

    class MotionListener implements MouseMotionListener {

    @Override
    public void mouseDragged(MouseEvent e) {
        for (int i = 0; i < size; i++) {
            for (int j = 0; j < size; j++) {
                if (e.getSource() == squares[i][j]) {
                    if (litSquares[i][j] == false) {
                        squares[i][j].setBackground(selected);
                        litSquares[i][j] = true;
                    } else {
                        squares[i][j].setBackground(backGround);
                        litSquares[i][j] = false;
                    }
                }
            }
        }
    }

    @Override
    public void mouseMoved(MouseEvent e) {
        // TODO Auto-generated method stub

    }

}

My JButton array is squares[][] and the litSquares[][] is a boolean map of what is selected currently, that I use when calculating the next step.

Any ideas on how to correct my MotionListener? I'm not understanding something about how to properly implement this class. All simple examples I find deal with drawing, but they all seem to track the Points where the cursor is dragged and update the pixels afterward. Is that something I will have to do somehow with my buttons?

***** Here is the MCVE, or at least as small as I could condense it. *****

package extraCredit;
import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.GridLayout;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.MouseEvent;
import java.awt.event.MouseMotionListener;

import javax.swing.*;
import javax.swing.border.Border;

public class MCVE extends JPanel {

    private static final long serialVersionUID = -8031881678612431401L;

    static JFrame frame;
    static JPanel grid;
    static JButton[][] squares;
    static boolean[][] litSquares, boardCopy;
    static int size, boardSize, tick = 1, goal = 100, rateIncrease = 10;
    ButtonListener listener = new ButtonListener();
    MotionListener mListerner = new MotionListener();
    Border noBorder = BorderFactory.createEmptyBorder();
    Color backGround = Color.BLUE;
    Color selected = Color.PINK;

    public MCVE(int size) {
        MCVE.size = size;
        squares = new JButton[size][size];
        litSquares = new boolean[size][size];
        grid = new JPanel(new GridLayout(size, size));
        for (int i = 0; i < size; i++) {
            for (int j = 0; j < size; j++) {
                squares[i][j] = new JButton();
                squares[i][j].addActionListener(listener);
                // squares[i][j].addMouseMotionListener(mListerner);
                squares[i][j].setBackground(backGround);
                squares[i][j].setBorder(noBorder);
                grid.add(squares[i][j]);
            }
        }

        frame = new JFrame();
        frame.setLayout(new BorderLayout());
        frame.add(grid, BorderLayout.CENTER);
        frame.setTitle("Life");
        if (25 * size < 525) {
            boardSize = 525;
        } else {
            boardSize = 25 * size;
        }
        frame.setSize(boardSize, boardSize);
        frame.setLocationRelativeTo(null);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setVisible(true);
    }

    class ButtonListener implements ActionListener {

        @Override
        public void actionPerformed(ActionEvent e) {
            for (int i = 0; i < size; i++) {
                for (int j = 0; j < size; j++) {
                    if (e.getSource() == squares[i][j]) {
                        if (litSquares[i][j] == false) {
                            squares[i][j].setBackground(selected);
                            litSquares[i][j] = true;
                        } else {
                            squares[i][j].setBackground(backGround);
                            litSquares[i][j] = false;
                        }
                    }
                }
            }
        }
    }

    class MotionListener implements MouseMotionListener {

        @Override
        public void mouseDragged(MouseEvent e) {
            for (int i = 0; i < size; i++) {
                for (int j = 0; j < size; j++) {
                    if (e.getSource() == squares[i][j]) {
                        if (litSquares[i][j] == false) {
                            squares[i][j].setBackground(selected);
                            litSquares[i][j] = true;
                        } else {
                            squares[i][j].setBackground(backGround);
                            litSquares[i][j] = false;
                        }
                    }
                }
            }
        }

        @Override
        public void mouseMoved(MouseEvent e) {
            // TODO Auto-generated method stub
        }
    }

    public static void main(String args[]) {
        new MCVE(20);
    }
}

SOLUTION BELOW, Thanks to @MadProgrammer and @durron597 . Mad answered my other question that was hanging me up on this problem.

import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Component;
import java.awt.GridLayout;
import java.awt.event.MouseListener;
import java.awt.event.MouseMotionListener;
import java.util.HashSet;
import java.util.Set;

import javax.swing.*;

public class ChangesAttempt extends JPanel {
private static final long serialVersionUID = -8031881678612431401L;

static JFrame frame;
static JPanel grid;
static JLabel[][] squares;
static boolean[][] litSquares;
static int size, boardSize;
static boolean startDrag, origin;
static Set<Component> compList = new HashSet<Component>();
MouseEvent listener = new MouseEvent();
MotionListener mListerner = new MotionListener();
Color backGround = Color.BLUE;
Color selected = Color.PINK;

public ChangesAttempt(int size) {
    ChangesAttempt.size = size;
    squares = new JLabel[size][size];
    litSquares = new boolean[size][size];
    grid = new JPanel(new GridLayout(size, size));
    grid.addMouseMotionListener(mListerner);
    grid.addMouseListener(listener);
    setBoard();

    frame = new JFrame();
    frame.setLayout(new BorderLayout());
    frame.add(grid, BorderLayout.CENTER);
    frame.setTitle("ChangedLife");
    if (25 * size < 525)
        boardSize = 525;
    else
        boardSize = 25 * size;
    frame.setSize(boardSize, boardSize);
    frame.setLocationRelativeTo(null);
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    frame.setVisible(true);
}

class MouseEvent implements MouseListener {

    @Override
    public void mousePressed(java.awt.event.MouseEvent e) {
        startDrag = true;
    }

    @Override
    public void mouseClicked(java.awt.event.MouseEvent e) {
        Component source = e.getComponent().getComponentAt(e.getPoint());
        System.out.println("X = " +source.getX() + ", Y = " + source.getY());
        for (int i = 0; i < size; i++) {
            for (int j = 0; j < size; j++) {
                if (source == squares[i][j]) {
                    if (litSquares[i][j] == false) {
                        squares[i][j].setBackground(selected);
                        litSquares[i][j] = true;
                    } else {
                        squares[i][j].setBackground(backGround);
                        litSquares[i][j] = false;
                    }
                }
            }
        }
    }

    @Override
    public void mouseEntered(java.awt.event.MouseEvent e) {
    }

    @Override
    public void mouseExited(java.awt.event.MouseEvent e) {
    }

    @Override
    public void mouseReleased(java.awt.event.MouseEvent e) {
        compList.clear();
    }
}

class MotionListener implements MouseMotionListener {
    @Override
    public void mouseDragged(java.awt.event.MouseEvent e) {
        compList.add(e.getComponent().getComponentAt(e.getPoint()));
        updateBoard();
    }

    @Override
    public void mouseMoved(java.awt.event.MouseEvent e) {

    }
}

public void setBoard() {
    for (int i = 0; i < size; i++) {
        for (int j = 0; j < size; j++) {
            squares[i][j] = new JLabel();
            squares[i][j].setOpaque(true);
            squares[i][j].setBackground(backGround);
            grid.add(squares[i][j]);
        }
    }
}

public void updateBoard(){
    for (int i = 0; i < size; i++) {
        for (int j = 0; j < size; j++) {
            if (compList.contains(squares[i][j])) {
                if(startDrag){
                    startDrag = false;
                    origin = litSquares[i][j];
                }
                if (litSquares[i][j] == origin) {
                    if(origin)
                        squares[i][j].setBackground(backGround);
                    else
                        squares[i][j].setBackground(selected);
                    litSquares[i][j] = !litSquares[i][j];
                }
            }
        }
    }
}

class MyLabel extends JLabel {
    private static final long serialVersionUID = -1414933339546989142L;

}

public static void main(String args[]) {
    new ChangesAttempt(20);
}
}

I think the problem you're having is that you're trying to use JButton for something it's not designed to do. I would go with a much simpler control, like JLabel, so that you won't have all the extra functionality that JButton has.

Secondly, your litSquares array is clearly a Data Clump. You absolutely should be creating your own custom control (which can either extend JComponent or extend JLabel), depending on whether JLabel meets your needs, that contains the other information that should be paired with the control itself.

Thirdly, you might not want the state of the button to be a boolean, you probably want it to be an enum... like

public enum State {
    UNLIT, LIT, CURRENTLY_SELECTED;
}

so that you can see which components are being newly color-flipped.

Finally, as was mentioned in the comments, use public Component getComponentAt(int x, int y) to be able to figure out what the MouseMotionListener was dragged over, and use that to change the color.