Deleting row in a table with delete button in PHP

I have some data coming from database and I have a delete button in last column. I want to delete the row on which delete button is clicked. But whenever I click delete button only last row is deleted even if I click on first row.

My delete function:

if(isset($_POST['formDelete'])){
if(isset($_POST['quoteid']) && !empty($_POST['quoteid'])){
    require_once('config.php');
    $conn = new mysqli(DB_HOST, DB_USER, DB_PASSWORD, DB_DATABASE) 
    or die ('Cannot connect to db');
    $quoteid = $_POST['quoteid'];
    echo "DELETE FROM quotes WHERE quoteid =".$quoteid;
    $result = $conn->query("DELETE FROM quotes WHERE quoteid =".$quoteid);
}

Table code:

<form action="<?php $_PHP_SELF ?>" method="post">
<?php
    require_once('config.php');
    $conn = new mysqli(DB_HOST, DB_USER, DB_PASSWORD, DB_DATABASE) 
    or die ('Cannot connect to db');
    $result = $conn->query("SELECT quoteid, quote, cat.catid catid, cat.name cat, scat.catid catid, scat.name scat, author FROM quotes q
                           INNER JOIN category cat ON q.catid = cat.catid LEFT OUTER JOIN category scat ON q.subcatid = scat.catid order by quoteid");
    echo "<table border='1px solid black'>";
    echo "<tr><td><b>Category</b></td><td><b>Sub-Category</b></td><td><b>Quote</b></td><td><b>Author</b></td></tr>";
    while ($row = $result->fetch_assoc()) {
        echo "<tr><td>$row[cat]</td><td>$row[scat]</td><td><input type='hidden' id='quoteid' name='quoteid' value=$row[quoteid] />$row[quote]</td><td>$row[author]</td>
        <td><input type='submit' name='formDelete' id='formDelete' value='Delete' /></td></tr>";
    }
    echo "</table>";
?>
</form>

I am trying to delete on the basis of quoteid.

The idea is create a separate form for each delete button so the right value for quoteid will be send

<?php
    require_once('config.php');
    $conn = new mysqli(DB_HOST, DB_USER, DB_PASSWORD, DB_DATABASE) 
    or die ('Cannot connect to db');
    $result = $conn->query("SELECT quoteid, quote, cat.catid catid, cat.name cat, scat.catid catid, scat.name scat, author FROM quotes q
                           INNER JOIN category cat ON q.catid = cat.catid LEFT OUTER JOIN category scat ON q.subcatid = scat.catid order by quoteid");
?>
    <table border='1px solid black'>
        <tr>
            <td>
                <b>Category</b>
            </td>
            <td>
                <b>Sub-Category</b>
            </td>
            <td>
                <b>Quote</b>
            </td>
            <td>
                <b>Author</b>
            </td>
        </tr>

        <?php
        while ($row = $result->fetch_assoc()) {
            ?>
            <tr>
                <td>
                    <?php echo $row['cat']; ?>
                </td>
                <td>
                    <?php echo $row['scat']; ?>
                </td>
                <td>
                    <?php echo $row['quote']; ?>
                </td>
                <td>
                    <?php echo $row['author']; ?>
                </td>
                <td>
                    <form action="<?php echo $_SERVER['PHP_SELF']; ?>" method='post' style="display: inline;">
                        <input type='hidden' id='quoteid' name='quoteid' value="<?php echo $row['quoteid']; ?>" />
                        <input type='submit' name='formDelete' id='formDelete' value='Delete' />
                    </form>
                </td>
            </tr>
            <?php
        }
        ?>
    </table>