读取大文件(此示例用的是nginx日志文件,约4G),并将其 批量写入数据库的方法
因为读取nginx日志大文件并拆分成必要的列 比较耗时,所以插入100万条数据用时约1分钟,如果不计算的话,批量插入效率会更高
资源文件列表
.vs/ConsoleAppBatInsert/DesignTimeBuild/.dtbcache , 230
.vs/ConsoleAppBatInsert/v15/.suo , 30720
.vs/ConsoleAppBatInsert/v15/Server/sqlite3/db.lock , 0
.vs/ConsoleAppBatInsert/v15/Server/sqlite3/storage.ide , 438272
.vs/ConsoleAppBatInsert/v15/Server/sqlite3/storage.ide-shm , 32768
.vs/ConsoleAppBatInsert/v15/Server/sqlite3/storage.ide-wal , 4136512
ConsoleAppBatInsert/App.config , 234
ConsoleAppBatInsert/ConsoleAppBatInsert.csproj , 2580
ConsoleAppBatInsert/Program.cs , 5983
ConsoleAppBatInsert/Properties/AssemblyInfo.cs , 1362
ConsoleAppBatInsert/bin/Debug/ConsoleAppBatInsert.exe , 8704
ConsoleAppBatInsert/bin/Debug/ConsoleAppBatInsert.exe.config , 234
ConsoleAppBatInsert/bin/Debug/ConsoleAppBatInsert.pdb , 15872
ConsoleAppBatInsert/bin/Debug/ConsoleAppBatInsert.vshost.exe , 24224
ConsoleAppBatInsert/bin/Debug/ConsoleAppBatInsert.vshost.exe.config , 234
ConsoleAppBatInsert/bin/Debug/ConsoleAppBatInsert.vshost.exe.manifest , 490
ConsoleAppBatInsert/bin/Debug/ECS_nginx_225.log , 17623421
ConsoleAppBatInsert/obj/Debug/ConsoleAppBatInsert.csproj.CoreCompileInputs.cache , 42
ConsoleAppBatInsert/obj/Debug/ConsoleAppBatInsert.csproj.FileListAbsolute.txt , 1896
ConsoleAppBatInsert/obj/Debug/ConsoleAppBatInsert.csprojResolveAssemblyReference.cache , 1755
ConsoleAppBatInsert/obj/Debug/ConsoleAppBatInsert.exe , 8704
ConsoleAppBatInsert/obj/Debug/ConsoleAppBatInsert.pdb , 15872
ConsoleAppBatInsert/obj/Debug/DesignTimeResolveAssemblyReferencesInput.cache , 6610
ConsoleAppBatInsert/obj/Debug/TemporaryGeneratedFile_036C0B5B-1481-4323-8D20-8F5ADCB23D92.cs , 0
ConsoleAppBatInsert/obj/Debug/TemporaryGeneratedFile_5937a670-0e60-4077-877b-f7221da3dda1.cs , 0
ConsoleAppBatInsert/obj/Debug/TemporaryGeneratedFile_E7A71F73-0F8D-4B9B-B56E-8E70B10BC5D3.cs , 0
ConsoleAppBatInsert.sln , 1026
EbgLog.sql , 1016
