Updating multiple rows of single table

java, mysql

Solution

If every row should get a different value that cannot be derived from the existing data in the database, there is not much you can do to optimize the overall complexity. So do not expect too much wonders.

That said, you should start using prepared statements and batching:

public void updateRank(Map<Integer,Double> map){
    Iterator<Entry<Integer, Double>> it = map.entrySet().iterator();
    String query = "";
    int i = 0;

    Connection connection = getConnection(); // get the DB connection from somewhere
    PreparedStatement stmt = connection.prepareStatement("update profile set rank = ? where profileId = ?");

    while (it.hasNext()) {
        Map.Entry<Integer,Double> pairs = (Map.Entry<Integer,Double>)it.next();
        stmt.setInt(1, pairs.getValue());
        stmt.setDouble(2, pairs.getKey());
        stmt.addBatch(); // this will just collect the data values
        it.remove();
    }       
    stmt.executeBatch(); // this will actually execute the updates all in one
}

What this does:

- the prepared statement causes the SQL parser to only parse the SQL once

- the batching minimizes the client-server-roundtrips so that not one for every update

- the communication between client and server is minimized because the SQL is only transmitted once and the data is collected and sent as a packet (or at least fewer packets)

In addition:

- Please check if the database column `profileId` is using an index so that looking up the respective row is fast enough

- You could check if your connection is set to auto-commit. If so try to disable auto-commit and explicitly commit the transaction after all rows are updated. This way the single update operations could be faster as well.

Problem

I need to update every row of a table having more then 60k rows. Currently I'm doing it like this: ``` public void updateRank(Map<Integer, Double> map) { Iterator<Map.Entry<Integer, Double>> it = map.entrySet().iterator(); while (it.hasNext()) { Map.Entry<Integer, Double> pairs = (Map.Entry<Integer, Double>) it.next(); String query = "update profile set rank = " + pairs.getValue() + " where profileId = " + pairs.getKey(); DBUtil.update(query); it.remove(); } } ``` This method alone took around 20+ mins to complete, hitting the database for each row(60k) is what i think the reason here.(though i'm using dbcp for connecton pooling, with 50 maximum active connections) It'd be great if i'd be able to update rows with single database hit. Is that Possible ? How ? Or any other way to improve timing here ?

Original source

Related problems