{"id":177,"date":"2024-06-25T01:24:34","date_gmt":"2024-06-25T01:24:34","guid":{"rendered":"https:\/\/www.kwwd.co.uk\/blog\/?p=177"},"modified":"2024-06-25T01:24:34","modified_gmt":"2024-06-25T01:24:34","slug":"split-a-large-mysql-table-into-multiple-sql-files","status":"publish","type":"post","link":"https:\/\/www.kwwd.co.uk\/blog\/split-a-large-mysql-table-into-multiple-sql-files\/","title":{"rendered":"Split A Large MySQL Table Into Multiple SQL Files"},"content":{"rendered":"<p>If you have a large database table that is causing your import to fail, you can split the table out into smaller more manageable chunks to import.<\/p>\n<p>Here&#8217;s a PHP script to create multiple SQL files:<\/p>\n<pre><code class=\"language-php line-numbers\">\r\n&lt;?php\r\n\r\n$servername = '192.0.0.0'; \/\/ IP or Host name\r\n$username = 'username';\r\n$password = 'password';\r\n$dbname = 'databasename';\r\n\r\n$chunkSize = 10000; \/\/ Number of rows per SQL file\r\n\r\ntry {\r\n    \/\/ Connect to the database\r\n    $pdo = new PDO(\"mysql:host=$servername;dbname=$dbname\", $username, $password);\r\n    $pdo-&gt;(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);\r\n\r\n    \/\/ Get the total number of rows in the table\r\n    $stmt = $pdo-&gt;query(\"SELECT COUNT(*) FROM $table\");\r\n    $totalRows = $stmt-&gt;fetchColumn();\r\n\r\n    $numFiles = ceil($totalRows \/ $chunkSize);\r\n\r\n    for ($i = 0; $i &lt; $numFiles; $i++) {\r\n        $offset = $i * $chunkSize;\r\n        $sqlFileName = \"export_chunk_{$i}.sql\";\r\n        $sqlFile = fopen($sqlFileName, 'w');\r\n\r\n        if ($sqlFile) {\r\n            $stmt = $pdo-&gt;prepare(\"SELECT * FROM $table LIMIT :limit OFFSET :offset\");\r\n            $stmt-&gt;bindValue(':limit', $chunkSize, PDO::PARAM_INT);\r\n            $stmt-&gt;bindValue(':offset', $offset, PDO::PARAM_INT);\r\n            $stmt-&gt;execute();\r\n\r\n            \/\/ Fetch data and write to the SQL file\r\n            while ($row = $stmt-&gt;fetch(PDO::FETCH_ASSOC)) {\r\n                $columns = implode(\", \", array_keys($row));\r\n                $values = implode(\", \", array_map(function($value) {\r\n                    return \"'\" . addslashes($value) . \"'\";\r\n                }, array_values($row)));\r\n                $sql = \"INSERT INTO $table ($columns) VALUES ($values);\\n\";\r\n                fwrite($sqlFile, $sql);\r\n            }\r\n\r\n            fclose($sqlFile);\r\n            echo \"Created $sqlFileName\\n\";\r\n        } else {\r\n            throw new Exception(\"Unable to open file $sqlFileName for writing.\");\r\n        }\r\n    }\r\n\r\n    echo \"All files created successfully.\\n\";\r\n\r\n} catch (PDOException $e) {\r\n    echo \"Database error: \" . $e-&gt;getMessage() . \"\\n\";\r\n} catch (Exception $e) {\r\n    echo \"Error: \" . $e-&gt;getMessage() . \"\\n\";\r\n}\r\n?&gt;\r\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>If you have a large database table that is causing your import to fail, you can split the table out into smaller more manageable chunks [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[24,9],"class_list":["post-177","post","type-post","status-publish","format-standard","hentry","category-php","tag-mysql","tag-php"],"_links":{"self":[{"href":"https:\/\/www.kwwd.co.uk\/blog\/wp-json\/wp\/v2\/posts\/177","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kwwd.co.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kwwd.co.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kwwd.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kwwd.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=177"}],"version-history":[{"count":0,"href":"https:\/\/www.kwwd.co.uk\/blog\/wp-json\/wp\/v2\/posts\/177\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.kwwd.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=177"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kwwd.co.uk\/blog\/wp-json\/wp\/v2\/categories?post=177"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kwwd.co.uk\/blog\/wp-json\/wp\/v2\/tags?post=177"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}