Flash Attention OFF:

| CPU | Config | Model | Th | FA | Enc. | Dec. | Bch5 | PP | Commit | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | M2 ULTRA | METAL | tiny | 1 | 0 | 10.15 | 1.20 | 0.36 | 0.01 | dc8dda60 | | M2 ULTRA | METAL | tiny-q5_0 | 1 | 0 | 10.21 | 1.15 | 0.39 | 0.01 | dc8dda60 | | M2 ULTRA | METAL | tiny-q5_1 | 1 | 0 | 9.26 | 1.15 | 0.38 | 0.01 | dc8dda60 | | M2 ULTRA | METAL | tiny-q8_0 | 1 | 0 | 9.00 | 1.12 | 0.37 | 0.01 | dc8dda60 | | M2 ULTRA | METAL | base | 1 | 0 | 15.77 | 1.73 | 0.45 | 0.02 | dc8dda60 | | M2 ULTRA | METAL | base-q5_0 | 1 | 0 | 16.90 | 1.63 | 0.44 | 0.02 | dc8dda60 | | M2 ULTRA | METAL | base-q5_1 | 1 | 0 | 16.93 | 1.64 | 0.44 | 0.02 | dc8dda60 | | M2 ULTRA | METAL | base-q8_0 | 1 | 0 | 16.13 | 1.63 | 0.43 | 0.02 | dc8dda60 | | M2 ULTRA | METAL | small | 1 | 0 | 45.15 | 3.45 | 0.92 | 0.05 | dc8dda60 | | M2 ULTRA | METAL | small-q5_0 | 1 | 0 | 50.63 | 3.36 | 0.94 | 0.06 | dc8dda60 | | M2 ULTRA | METAL | small-q5_1 | 1 | 0 | 50.56 | 3.36 | 0.94 | 0.06 | dc8dda60 | | M2 ULTRA | METAL | small-q8_0 | 1 | 0 | 47.52 | 3.20 | 0.92 | 0.05 | dc8dda60 | | M2 ULTRA | METAL | medium | 1 | 0 | 122.55 | 7.38 | 1.95 | 0.12 | dc8dda60 | | M2 ULTRA | METAL | medium-q5_0 | 1 | 0 | 140.61 | 6.73 | 2.02 | 0.14 | dc8dda60 | | M2 ULTRA | METAL | medium-q5_1 | 1 | 0 | 140.48 | 6.76 | 2.04 | 0.14 | dc8dda60 | | M2 ULTRA | METAL | medium-q8_0 | 1 | 0 | 131.00 | 6.57 | 1.96 | 0.13 | dc8dda60 | | M2 ULTRA | METAL | medium-dis | 1 | 0 | 110.85 | 1.00 | 0.24 | 0.02 | dc8dda60 | | M2 ULTRA | METAL | large-v2 | 1 | 0 | 222.28 | 10.96 | 3.03 | 0.21 | dc8dda60 | | M2 ULTRA | METAL | large-v2-q5_0 | 1 | 0 | 258.64 | 9.79 | 3.04 | 0.25 | dc8dda60 | | M2 ULTRA | METAL | large-v2-q5_1 | 1 | 0 | 258.32 | 9.87 | 3.05 | 0.24 | dc8dda60 | | M2 ULTRA | METAL | large-v2-q8_0 | 1 | 0 | 236.55 | 9.61 | 2.87 | 0.23 | dc8dda60 | | M2 ULTRA | METAL | large-v2-dis | 1 | 0 | 199.84 | 1.14 | 0.27 | 0.02 | dc8dda60 | | M2 ULTRA | METAL | large-v3-turbo | 1 | 0 | 201.52 | 1.77 | 0.45 | 0.03 | dc8dda60 | | M2 ULTRA | METAL | large-v3-turbo-q5_0 | 1 | 0 | 233.14 | 1.56 | 0.47 | 0.04 | dc8dda60 | | M2 ULTRA | METAL | large-v3-turbo-q8_0 | 1 | 0 | 214.23 | 1.53 | 0.44 | 0.04 | dc8dda60 |

What's Changed

• docs : add xcframework section to README.md [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/2997
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/2992
• whisper.wasm : fix unknown language issue by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3000
• examples : update server.py to match github pages app [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3004
• rename : ggerganov -> ggml-org by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3005
• whisper : fix "bench-all outputs an invalid result on larger models" by @fujimotos in https://github.com/ggml-org/whisper.cpp/pull/3002
• tests : add script to benchmark whisper.cpp on LibriSpeech corpus by @fujimotos in https://github.com/ggml-org/whisper.cpp/pull/2999
• ruby : Change homepage URI in Ruby gemspec by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3007
• fix dead link to models in readme by @gregsadetsky in https://github.com/ggml-org/whisper.cpp/pull/3006
• Update uri.rb by @Olli in https://github.com/ggml-org/whisper.cpp/pull/3016
• Update ruby_whisper_params.c by @Olli in https://github.com/ggml-org/whisper.cpp/pull/3022
• xcf : use check for visionos build version by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3021
• Fix README.md by @ekaitz-zarraga in https://github.com/ggml-org/whisper.cpp/pull/3024
• docs : document how to use 'WHISPER_FFMPEG' build option by @fujimotos in https://github.com/ggml-org/whisper.cpp/pull/3029
• whisper : reduce delta_min from 1000ms to 100ms by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3028
• support max_context api for addon.node by @buxuku in https://github.com/ggml-org/whisper.cpp/pull/3025
• Update README.md to note newer NVIDIA GPUs by @jeffklassen in https://github.com/ggml-org/whisper.cpp/pull/3031
• ruby: use CMake in build process by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3043
• examples : add FFmpeg v7.0 support to ffmpeg-transcode.cpp by @fujimotos in https://github.com/ggml-org/whisper.cpp/pull/3038
• feat: Add no-context option to server by @sachaarbonel in https://github.com/ggml-org/whisper.cpp/pull/3045
• ruby : make Ruby bindings installed with build options by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3056
• examples : add HEAPU8 to exported runtime methods by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3062
• ci : disable freeBSD job in build.yml by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3064
• coreml : set convert_to="mlprogram" in convert by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3060
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3071
• ci : enable bindings java job by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3070
• ruby : add encoder begin callback related methods by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3076
• Fix deprecated FFmpeg functions by @Podre-Henrique in https://github.com/ggml-org/whisper.cpp/pull/3073
• Add Moore Threads GPU support and update GitHub workflow for MUSA build by @yeahdongcn in https://github.com/ggml-org/whisper.cpp/pull/3069
• ci : disable publishing of java binding [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3086
• talk-llama : sync llama.cpp by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3084
• whisper : remove empty .gitmodules file [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3085
• feat: expose language detection probabilities to server example by @sachaarbonel in https://github.com/ggml-org/whisper.cpp/pull/3044
• whisper : fix grammar advance stack warning by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3087
• ggml : suppress Windows compiler warnings by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3075
• make : fix samples glob pattern by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3100
• ruby : ignore "Downloading" output in test_log_suppress by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3106
• server : add --no-gpu option to print usage output by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3098
• whisper : add check that target name exists by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3103
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3108
• ruby : refine HTTP cache feature by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3109
• feat(server): bump httplib.h to version 0.20.0 by @sachaarbonel in https://github.com/ggml-org/whisper.cpp/pull/3101
• removed stdc++fs by @JaredTweed in https://github.com/ggml-org/whisper.cpp/pull/3097
• docs : Update cli documentation by @antimatter96 in https://github.com/ggml-org/whisper.cpp/pull/3102
• cli : Support "-" for stdout like stdin by @danielzgtg in https://github.com/ggml-org/whisper.cpp/pull/3050
• feat(server): Improve HTTP Connection Abort Handling by @sachaarbonel in https://github.com/ggml-org/whisper.cpp/pull/3112
• whisper: remove MSVC warnings pragmas by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3090
• ci : add zip extension to xcframework artifact name by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3120
• ci : zip windows artifacts for release uploading by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3124
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3125
• ci : add bindings-java jar artifact to release by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3126
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3127
• stream.wasm : add HEAPU8 to exported runtime methods by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3130
• whisper : deprecate WHISPER_CCACHE CMake option by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3131
• wasm : add note about worker.js file generation [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3133
• examples : add HEAPU8 to all of the exported runtime methods by @enesgrahovac in https://github.com/ggml-org/whisper.cpp/pull/3134
• ruby : omit test_build_options locally by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3132
• ruby : test extra build options only when env var specified by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3136
• examples : update link to Paul Tol's color scheme [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3140
• Update Readme to recommend same Openvino as Python tools by @peardox in https://github.com/ggml-org/whisper.cpp/pull/3138
• cli : print color scheme info for --print-colors by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3141
• docs : fix -owts flag typo karaoke section [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3142
• whisper : remove dummy commit comment [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3143
• vad : add initial Voice Activity Detection (VAD) support by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3065
• docs: Update README.md for whisper.objc app by @tomer-schles in https://github.com/ggml-org/whisper.cpp/pull/2569
• vad : remove shortform for --vad option in cli.cpp by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3145
• examples : add VAD speech segments example by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3147
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3148
• server : add --flash-attn usage output by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3152
• vad : add download-vad-model scripts by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3149
• examples : add --print-confidence option to cli by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3150
• whisper : add build_*/ to .gitignore [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3157
• vad : store VAD context in whisper_state by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3156
• vad : return early if no vad segments are detected by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3158
• examples : add vad-speech-segments to win warns [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3170
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3171
• ruby : add GGML_SYCL_DNN option to ruby bindings by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3172
• Fix CMakeLists.txt to handle deprecated gpu Warnings by @jugal-sheth in https://github.com/ggml-org/whisper.cpp/pull/3163
• whisper : remove redundant assignments by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3178
• docs: fix very minor typo in README by @alpaim in https://github.com/ggml-org/whisper.cpp/pull/3179
• docs : add VAD model download instructions [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3180
• feat(server): Add k6 Load Testing Script by @sachaarbonel in https://github.com/ggml-org/whisper.cpp/pull/3175
• ci : use dynamic libopenblas.dll for window-blas by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3177
• docs : Fix VAD section heading levels in README by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3186
• Quickfix for swedish umlauts + expose model inference settings in talk-llama.cpp by @matteng1 in https://github.com/ggml-org/whisper.cpp/pull/3187
• node : enable no_prints to suppress all output by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3189
• docs : convert README_sycl.md to utf8 format [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3191
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3193
• ci : update windows artifact uploads steps by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3192
• tests : add a new benchmark test for long-form audio by @fujimotos in https://github.com/ggml-org/whisper.cpp/pull/3185
• Install shared libs when using GGML_BACKEND_DL by @peardox in https://github.com/ggml-org/whisper.cpp/pull/3195
• ruby : add VAD support, migration to Ruby's newer API by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3197
• whisper : remove whisper_load_backends function by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3196
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3202
• ggml : Fix backtrace breaking Windows build by @danielzgtg in https://github.com/ggml-org/whisper.cpp/pull/3203
• ruby : handle build options on installation by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3206
• vad : revisit timestamp alignment/mapping by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3173
• ruby : add Core ML support by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3214
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3215
• node : add language detection support by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3190
• ci : use mirrors.kernel.org for Ubuntu packages by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3220
• musa: correct MUSA SDK rc4.0.1 download URL by @yeahdongcn in https://github.com/ggml-org/whisper.cpp/pull/3217
• fix: apply whisperParams in fullTranscribeWithTime by @Jo4sDev in https://github.com/ggml-org/whisper.cpp/pull/3201
• ci : add mirror for ports.ubuntu.com (ARM packages) by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3221
• ruby : Add parallel transcription support by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3222
• devops: add main-intel dockerfile by @kwaa in https://github.com/ggml-org/whisper.cpp/pull/3229
• ci(docker): build and publish main-intel image by @kwaa in https://github.com/ggml-org/whisper.cpp/pull/3231
• ruby : output format by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3237
• sync : whisper.cpp by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3239
• ruby : add cleaning of library names in dependencies by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3241
• ci : update windows runner to windows-2022 by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3242
• ruby : add .gitignore entries for ext directory by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3245
• cli : fix short name conflict for vad options [no ci] by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3247
• server : add Voice Activity Detection (VAD) support by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3246
• whisper : fix VAD processing for skipped audio segments by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3230
• feat(server): graceful shutdown, atomic server state, and health endpoint Improvements by @sachaarbonel in https://github.com/ggml-org/whisper.cpp/pull/3243
• let people select the model the wanna use with yt-wsp.sh using a second argument by @w1redch4d in https://github.com/ggml-org/whisper.cpp/pull/3251
• examples : set the C++ standard to C++17 for server by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3261
• whisper : clear result_all if vad_samples is empty by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3262
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3264
• examples : add stereo to mono conversion in read_audio_data by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3266
• cmake : fix android build by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3265
• android : update CMakeLists.txt to use FetchContent for ggml by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3268
• sync : ggml by @ggerganov in https://github.com/ggml-org/whisper.cpp/pull/3272
• ruby : specify Apple frameworks explicitly on build by @KitaitiMakoto in https://github.com/ggml-org/whisper.cpp/pull/3270
• ci : add apt-get clean to musa Dockerfile by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3275
• whisper-command: write commands to text file by @aaron-ang in https://github.com/ggml-org/whisper.cpp/pull/3273
• chore: add .gitignore entries for OpenVINO support by @mochiya98 in https://github.com/ggml-org/whisper.cpp/pull/3276
• ci : reduce musa image size by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3277
• backport CoreML features to macos < 14 by @glaszig in https://github.com/ggml-org/whisper.cpp/pull/3255
• feat: update java example by @Jo4sDev in https://github.com/ggml-org/whisper.cpp/pull/3281
• ci : enable main-cuda build by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3282
• stream : add nullptr check of whisper_context by @danbev in https://github.com/ggml-org/whisper.cpp/pull/3283

New Contributors

• @fujimotos made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3002
• @gregsadetsky made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3006
• @Olli made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3016
• @ekaitz-zarraga made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3024
• @jeffklassen made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3031
• @Podre-Henrique made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3073
• @yeahdongcn made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3069
• @JaredTweed made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3097
• @antimatter96 made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3102
• @danielzgtg made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3050
• @enesgrahovac made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3134
• @peardox made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3138
• @tomer-schles made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/2569
• @jugal-sheth made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3163
• @alpaim made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3179
• @matteng1 made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3187
• @Jo4sDev made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3201
• @kwaa made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3229
• @w1redch4d made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3251
• @aaron-ang made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3273
• @mochiya98 made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3276
• @glaszig made their first contribution in https://github.com/ggml-org/whisper.cpp/pull/3255

Full Changelog: https://github.com/ggml-org/whisper.cpp/compare/v1.7.5...v1.7.6