Skip to content

Temp #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 19 commits into from
Nov 15, 2024
Merged

Temp #19

merged 19 commits into from
Nov 15, 2024

Conversation

apicalshark
Copy link
Owner

jeffbolznv and others added 18 commits November 11, 2024 18:13
@jeffbolznv
vulkan: Throttle the number of shader compiles during the build step. (…
54ef9cf 
…ggml-org#10222)

Fixes ggml-org#9582

Spawning too many concurrent copies of glslc leads to "Failed to create pipes"
errors on Linux. This change applies the same throttling we use for
multithreaded pipeline creation.
@jeffbolznv
vulkan: Optimize contiguous copies (ggml-org#10254)
80dd7ff 
* tests: Fix memory bandwidth calculation for perf tests

Add a flops calculation for flash attention.

Add one GGML_OP_CPY perf test.

* vulkan: Optimize contiguous copies

Add a variant of the copy shader for when the tensors are contiguous. Avoid
the complex addressing calculations, and do four elements per invocation
to hide some other overhead.

Apply similar changes to the scale shader, since scale is always contiguous.

Add a "progress bar" for shader compiles.
@Alcpz
sycl : Fixes to broken builds and test-backend-ops (ggml-org#10257)
2e82ffa 
* Fixes broken build for the SYCL CUDA backend caused by non-explicit gemm call in outprod (merged in with RWKV6 in
Optimize RWKV6 Operator Naming and Implement Multi-core CPU/ SYCL Acceleration ggml-org#10133)

* Marks permuted MUL_MAT as unsupported to be able to run test-backend-ops

* Fixes asserts in norm to fix debug builds.
@mofosyne
metadata: Detailed Dataset Authorship Metadata (ggml-org#8875)
a0ec17b 
Converter script can now read these two fields as a detailed base model and dataset source.
This was done so that it will be easier for Hugging Face to integrate detailed metadata as needed.

 -  base_model_sources (List[dict], optional)
 -  dataset_sources (List[dict], optional)

Dataset now represented as:

 - general.dataset.count
 - general.dataset.{id}.name
 - general.dataset.{id}.author
 - general.dataset.{id}.version
 - general.dataset.{id}.organization
 - general.dataset.{id}.description
 - general.dataset.{id}.url
 - general.dataset.{id}.doi
 - general.dataset.{id}.uuid
 - general.dataset.{id}.repo_url

This also adds to base model these metadata:

 - general.base_model.{id}.description
@jhen0409
server : fix incorrect res in validate_model_chat_template (ggml-org#…
0e712a5 
…10272)

* server : fix validate_model_chat_template

* server : fix chat res
@z80maniac
server : add missing docs (ggml-org#10269)
ff7fb67
@xuegao-tzx
docs : update bindings list (ggml-org#10261)
1ee9eea 
Signed-off-by: tianzixuan <tianzixuan335@hellobike.com>
@ggerganov
sync : ggml
5ea926d
@Xarbirus @ggerganov
llama : propagate the results of graph_compute (ggml-org#9525)
fb4a0ec 
* llama: propagating the results of `graph_compute` to the user interface

* llama: reverting kv_cache in case of failed compute

* llama: `llama_kv_cache_state` was removed, only the result of `llama_graph_compute` is returned

* llama: restore a kv_cache in case of failed computation

* llama: correct reverting of the entire batch.
also updates `llama_kv_cache_find_slot`, will correctly count the number of `used` cells for recurrent models

* llama: updated comments

* llama : add comments about KV cache state after error

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
@jeffbolznv
vulkan: Use macros to make the mat mul pipeline creation more concise (…
66798e4 
…ggml-org#10259)

Also add vk_matmul_pipeline2 to hold f16/f32 accumulator versions of a
pipeline. This isn't really used yet.
@jeffbolznv
vulkan: Optimize binary ops (ggml-org#10270)
af148c9 
Reuse the index calculations across all of src0/src1/dst. Add a shader
variant for when src0/src1 are the same dimensions and additional modulus
for src1 aren't needed. Div/mod are slow, so add "fast" div/mod that
have a fast path when the calculation isn't needed or can be done more
cheaply.
@ggerganov
speculative : fix out-of-bounds access (ggml-org#10289)
2a82891
@JohannesGaessler
CUDA: no -sm row for very small matrices (ggml-org#10185)
4a8ccb3
@slaren @ggerganov @yeahdongcn
ggml : build backends as libraries (ggml-org#10256)
ae8de6d 
* ggml : build backends as libraries

---------

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: R0CKSTAR <xiaodong.ye@mthreads.com>
@chaxu01 @slaren
backend cpu: add online flow for aarch64 Q4_0 GEMV/GEMM kernels (ggml…
1607a5e 
…-org#9921)

* backend-cpu: add online flow for aarch64 Q4_0 GEMV/GEMM kernels

---------

Co-authored-by: Diego Devesa <slarengh@gmail.com>
@Rbiessy
sycl: Use syclcompat::dp4a (ggml-org#10267)
5a54af4 
* sycl: Use syclcompat::dp4a

* Using the syclcompat version allow the compiler to optimize the
  operation with native function

* Update news section

* Update CI Windows oneAPI version to 2025.0

* Reword doc

* Call syclcompat::dp4a inside dpct::dp4a

This reverts commit 90cb61d.
@ggerganov
scripts : fix regex in sync [no ci]
4802ad3
@noemotiovon
cann: dockerfile and doc adjustment (ggml-org#10302)
231f936 
Co-authored-by: noemotiovon <noemotiovon@gmail.com>
@apicalshark apicalshark merged commit 4a91abe into master Nov 15, 2024
4 of 48 checks passed
@apicalshark apicalshark deleted the temp branch November 15, 2024 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Apple Metal build devops documentation Improvements or additions to documentation examples ggml Kompute nix Nvidia GPU python script server SYCL testing
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.