批量拆分一个文本文件

Question

我有这个批处理文件来拆分txt文件：

@echo off
for /f "tokens=1*delims=:" %%a in ('findstr /n "^" "PASSWORD.txt"') do for /f "delims=~" %%c in ("%%~b") do >"text%%a.txt" echo(%%c
pause

它可以工作，但它一行一行地拆分它。我如何让它每5000行拆分一次。提前感谢。

编辑：

I have just tried this:

@echo off
setlocal ENABLEDELAYEDEXPANSION
REM Edit this value to change the name of the file that needs splitting. Include the extension.
SET BFN=passwordAll.txt
REM Edit this value to change the number of lines per file.
SET LPF=50000
REM Edit this value to change the name of each short file. It will be followed by a number indicating where it is in the list.
SET SFN=SplitFile

REM Do not change beyond this line.

SET SFX=%BFN:~-3%

SET /A LineNum=0
SET /A FileNum=1

For /F "delims==" %%l in (%BFN%) Do (
SET /A LineNum+=1

echo %%l >> %SFN%!FileNum!.%SFX%

if !LineNum! EQU !LPF! (
SET /A LineNum=0
SET /A FileNum+=1
)

)
endlocal
Pause
exit

但我得到一个错误提示：＜code＞没有足够的存储空间来处理此命令。＜code＞

Answer 1

@echo off
setlocal EnableDelayedExpansion

findstr /N "^" PASSWORD.txt > temp.txt
set part=0
call :splitFile < temp.txt
del temp.txt
goto :EOF

:splitFile
set /A part+=1
(for /L %%i in (1,1,5000) do (
   set "line="
   set /P line=
   if defined line echo(!line:*:=!
)) >  text%part%.txt
if defined line goto splitFile
exit /B

如果输入文件没有空行，可以修改前面的方法以便运行得更快。

Answer 2

测试一下:例如，输入文件是< code >“file . txt”，输出文件是< code >“split file-5000 . txt”。

这使用了一个名为findrepl.bat的辅助批处理文件-下载地址：https://www.dropbox.com/s/rfdldmcb6vwi9xc/findrepl.bat

放置＜code＞findrepl。bat与批处理文件位于同一文件夹中或路径上。

@echo off
:: splits file.txt into 5000 line chunks. 
set chunks=5000

set /a s=1-chunks
:loop
set /a s=s+chunks
set /a e=s+chunks-1
echo %s% to %e%
call findrepl /o:%s%:%e% <"file.txt" >"splitfile-%e%.txt"
for %%b in ("splitfile-%e%.txt") do (if %%~zb EQU 0 del "splitfile-%e%.txt" & goto :done)
goto :loop
:done
pause

一个限制是文件中的行数，真正最大的行数是2^31-1，其中批处理数学最多。

Answer 3

这会给你一个基本的框架。根据需要进行调整

@echo off
    setlocal enableextensions disabledelayedexpansion

    set "nLines=5000"
    set "line=0"

    for /f "usebackq delims=" %%a in ("passwords.txt") do (
        set /a "file=line/%nLines%", "line+=1"
        setlocal enabledelayedexpansion
        for %%b in (!file!) do (
            endlocal
            >>"passwords_%%b.txt" echo(%%a
        )
    )

    endlocal

编辑

正如评论所指出的，4.3GB的文件很难管理。< code>for /f需要将整个文件加载到内存中，当文件在内存中转换为unicode时，所需的缓冲区是这个大小的两倍。

这是一个完全临时的解决方案。我还没有在这么高的文件中测试过它，但至少在理论上它应该可以工作（除非5000行需要大量的内存，这取决于行长）

而且，有了这样的文件，它会很慢。

@echo off
    setlocal enableextensions disabledelayedexpansion

    set "line=0"
    set "tempFile=%temp%\passwords.tmp"

    findstr /n "^" passwords.txt > "%tempFile%"
    for /f %%a in ('type passwords.txt ^| find /c /v "" ') do set /a "nFiles=%%a/5000"

    for /l %%a in (0 1 %nFiles%) do (
        set /a "e1=%%a*5", "e2=e1+1", "e3=e2+1", "e4=e3+1", "e5=e4+1"
        setlocal enabledelayedexpansion
        if %%a equ 0 (
            set "e=/c:"[1-9]:" /c:"[1-9][0-9]:" /c:"[1-9][0-9][0-9]:" /c:"!e2![0-9][0-9][0-9]:" /c:"!e3![0-9][0-9][0-9]:" /c:"!e4![0-9][0-9][0-9]:" /c:"!e5![0-9][0-9][0-9]:" "
        ) else (
            set "e=/c:"!e1![0-9][0-9][0-9]:" /c:"!e2![0-9][0-9][0-9]:" /c:"!e3![0-9][0-9][0-9]:" /c:"!e4![0-9][0-9][0-9]:" /c:"!e5![0-9][0-9][0-9]:" "
        )
        for /f "delims=" %%e in ("!e!") do (
            endlocal & (for /f "tokens=1,* delims=:" %%b in ('findstr /r /b %%e "%tempFile%"') do @echo(%%c)>passwords_%%a.txt
        )
    )

    del "%tempFile%" >nul 2>nul

    endlocal

再次编辑:前面的代码不能正确处理以冒号开头的行，因为它在< code>for命令中被用作分隔符，用于分隔行号和数据。

对于替代方案，仍然是纯批次，但仍然缓慢

@echo off
    setlocal enableextensions disabledelayedexpansion

    set "nLines=5000"
    set "line=0"
    for /f %%a in ('type passwords.txt^|find /c /v ""') do set "fileLines=%%a"

    < "passwords.txt" (for /l %%a in (1 1 %fileLines%) do (
        set /p "data="
        set /a "file=line/%nLines%", "line+=1"
        setlocal enabledelayedexpansion
        >>"passwords_!file!.txt" echo(!data!
        endlocal
    ))

    endlocal