Mercurial > vec

annotate src/impl/x86/avx512f.c @ 28:c6c99ab1088a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
*: add min/max functions and a big big refactor (again) agh, this time I added a few more implementations (and generally made the code just a little faster...)
author Paper <paper@tflc.us>
date Thu, 24 Apr 2025 00:54:02 -0400
parents e49e70f7012f
children bf6ad516f1e6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
1 /**
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
2 * vec - a tiny SIMD vector library in C99
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
3 *
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
4 * Copyright (c) 2024 Paper
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
5 *
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
6 * Permission is hereby granted, free of charge, to any person obtaining a copy
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
7 * of this software and associated documentation files (the "Software"), to deal
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
8 * in the Software without restriction, including without limitation the rights
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
9 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
10 * copies of the Software, and to permit persons to whom the Software is
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
11 * furnished to do so, subject to the following conditions:
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
12 *
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
13 * The above copyright notice and this permission notice shall be included in all
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
14 * copies or substantial portions of the Software.
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
15 *
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
16 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
17 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
18 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
19 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
20 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
21 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
22 * SOFTWARE.
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
23 **/
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
24
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
25 #include "vec/impl/x86/avx512f.h"
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
26
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
27 #include <immintrin.h>
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
28
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
29 /* ------------------------------------------------------------------------ */
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
30
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
31 #define VEC_AVX512F_MINMAX_TEMPLATE(SIGN, BITS, SIZE, INTLSIGN, OP) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
32 VEC_FUNC_IMPL v##SIGN##int##BITS##x##SIZE v##SIGN##int##BITS##x##SIZE##_avx512f_##OP(v##SIGN##int##BITS##x##SIZE vec1, v##SIGN##int##BITS##x##SIZE vec2) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
33 { \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
34 union v##SIGN##int##BITS##x##SIZE##_impl_data *vec1d = (union v##SIGN##int##BITS##x##SIZE##_impl_data *)&vec1; \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
35 union v##SIGN##int##BITS##x##SIZE##_impl_data *vec2d = (union v##SIGN##int##BITS##x##SIZE##_impl_data *)&vec2; \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
36 \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
37 vec1d->avx512f = _mm512_##OP##_ep##INTLSIGN##BITS(vec1d->avx512f, vec2d->avx512f); \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
38 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
39 return vec1d->vec; \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
40 }
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
41
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
42 #define VEC_AVX512F_MINMAX_32x16(OP) VEC_AVX512F_MINMAX_TEMPLATE( , 32, 16, i, OP)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
43 #define VEC_AVX512F_MINMAX_u32x16(OP) VEC_AVX512F_MINMAX_TEMPLATE(u, 32, 16, u, OP)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
44 #define VEC_AVX512F_MINMAX_64x8(OP) VEC_AVX512F_MINMAX_TEMPLATE( , 64, 8, i, OP)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
45 #define VEC_AVX512F_MINMAX_u64x8(OP) VEC_AVX512F_MINMAX_TEMPLATE(u, 64, 8, u, OP)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
46
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
47 #define VEC_AVX512F_STRUCT_MINMAX_32x16(OP, SIGN) v##SIGN##int32x16_avx512f_##OP
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
48 #define VEC_AVX512F_STRUCT_MINMAX_64x8(OP, SIGN) v##SIGN##int64x8_avx512f_##OP
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
49
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
50 /* ------------------------------------------------------------------------ */
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
51
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
52 #define VEC_AVX512F_OPERATION_EX(name, op, sign, bits, size, secondsign) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
53 VEC_FUNC_IMPL v##sign##int##bits##x##size v##sign##int##bits##x##size##_avx512f_##name(v##sign##int##bits##x##size vec1, v##secondsign##int##bits##x##size vec2) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
54 { \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
55 union v##sign##int##bits##x##size##_impl_data *vec1d = (union v##sign##int##bits##x##size##_impl_data *)&vec1; \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
56 union v##secondsign##int##bits##x##size##_impl_data *vec2d = (union v##secondsign##int##bits##x##size##_impl_data *)&vec2; \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
57 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
58 vec1d->avx512f = _mm512_##op##_epi##bits(vec1d->avx512f, vec2d->avx512f); \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
59 \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
60 return vec1d->vec; \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
61 }
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
62
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
63 #define VEC_AVX512F_OPERATION(name, op, sign, bits, size) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
64 VEC_AVX512F_OPERATION_EX(name, op, sign, bits, size, sign)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
65
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
66 #define VEC_AVX512F_OPERATION_SHIFT(name, op, sign, bits, size) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
67 VEC_AVX512F_OPERATION_EX(name, op, sign, bits, size, u)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
68
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
69 #define VEC_AVX512F_ADD_32x16(sign) VEC_AVX512F_OPERATION(add, add, sign, 32, 16)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
70 #define VEC_AVX512F_ADD_64x8(sign) VEC_AVX512F_OPERATION(add, add, sign, 64, 8)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
71
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
72 #define VEC_AVX512F_SUB_32x16(sign) VEC_AVX512F_OPERATION(sub, sub, sign, 32, 16)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
73 #define VEC_AVX512F_SUB_64x8(sign) VEC_AVX512F_OPERATION(sub, sub, sign, 64, 8)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
74
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
75 #define VEC_AVX512F_MUL_32x16(sign) VEC_AVX512F_OPERATION(mul, mullo, sign, 32, 16)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
76 #define VEC_AVX512F_MUL_64x8(sign) VEC_AVX512F_OPERATION(mul, mullox, sign, 64, 8)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
77
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
78 #define VEC_AVX512F_LSHIFT_32x16(sign) VEC_AVX512F_OPERATION_SHIFT(lshift, sllv, sign, 32, 16)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
79 #define VEC_AVX512F_LSHIFT_64x8(sign) VEC_AVX512F_OPERATION_SHIFT(lshift, sllv, sign, 64, 8)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
80
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
81 #define VEC_AVX512F_XRSHIFT(name, bits, size, sign, aORl) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
82 VEC_AVX512F_OPERATION_SHIFT(name, sr##aORl##v, sign, bits, size)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
83
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
84 /* always the same, regardless of signedness */
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
85 #define VEC_AVX512F_LRSHIFT_32x16(sign) VEC_AVX512F_XRSHIFT(lrshift, 32, 16, sign, l)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
86 #define VEC_AVX512F_LRSHIFT_64x8(sign) VEC_AVX512F_XRSHIFT(lrshift, 64, 8, sign, l)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
87
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
88 #define VEC_AVX512F_RSHIFT_32x16(sign) VEC_AVX512F_XRSHIFT(rshift, 32, 16, sign, a)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
89 #define VEC_AVX512F_RSHIFT_64x8(sign) VEC_AVX512F_XRSHIFT(rshift, 64, 8, sign, a)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
90
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
91 #define VEC_AVX512F_uRSHIFT_32x16(sign) VEC_AVX512F_XRSHIFT(rshift, 32, 16, sign, l)
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
92 #define VEC_AVX512F_uRSHIFT_64x8(sign) VEC_AVX512F_XRSHIFT(rshift, 64, 8, sign, l)
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
93
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
94 /* ok */
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
95 #define VEC_AVX512F_STRUCT_OPERATION_32x16(OP, SIGN) v##SIGN##int32x16_avx512f_##OP
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
96 #define VEC_AVX512F_STRUCT_OPERATION_64x8(OP, SIGN) v##SIGN##int64x8_avx512f_##OP
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
97
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
98 /* ------------------------------------------------------------------------ */
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
99
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
100 #define VEC_AVX512F_DEFINE_OPERATIONS_SIGN(sign, bits, size) \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
101 union v##sign##int##bits##x##size##_impl_data { \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
102 v##sign##int##bits##x##size vec; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
103 __m512i avx512f; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
104 }; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
105 \
24
e49e70f7012f impl/x86: add static assertions for alignment and size
Paper <paper@tflc.us>
parents: 23
diff changeset
106 VEC_STATIC_ASSERT(VEC_ALIGNOF(__m512i) <= VEC_ALIGNOF(v##sign##int##bits##x##size), "vec: v" #sign "int" #bits "x" #size " alignment needs to be expanded to fit intrinsic type size"); \
e49e70f7012f impl/x86: add static assertions for alignment and size
Paper <paper@tflc.us>
parents: 23
diff changeset
107 VEC_STATIC_ASSERT(sizeof(__m512i) <= sizeof(v##sign##int##bits##x##size), "vec: v" #sign "int" #bits "x" #size " needs to be expanded to fit intrinsic type size"); \
e49e70f7012f impl/x86: add static assertions for alignment and size
Paper <paper@tflc.us>
parents: 23
diff changeset
108 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
109 VEC_FUNC_IMPL v##sign##int##bits##x##size v##sign##int##bits##x##size##_avx512f_load_aligned(const vec_##sign##int##bits in[size]) \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
110 { \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
111 union v##sign##int##bits##x##size##_impl_data vec; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
112 vec.avx512f = _mm512_load_si512((const __m512i *)in); \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
113 return vec.vec; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
114 } \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
115 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
116 VEC_FUNC_IMPL v##sign##int##bits##x##size v##sign##int##bits##x##size##_avx512f_load(const vec_##sign##int##bits in[size]) \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
117 { \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
118 union v##sign##int##bits##x##size##_impl_data vec; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
119 vec.avx512f = _mm512_loadu_si512((const __m512i *)in); \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
120 return vec.vec; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
121 } \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
122 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
123 VEC_FUNC_IMPL void v##sign##int##bits##x##size##_avx512f_store_aligned(v##sign##int##bits##x##size vec, vec_##sign##int##bits out[size]) \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
124 { \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
125 _mm512_store_si512((__m512i *)out, ((union v##sign##int##bits##x##size##_impl_data *)&vec)->avx512f); \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
126 } \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
127 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
128 VEC_FUNC_IMPL void v##sign##int##bits##x##size##_avx512f_store(v##sign##int##bits##x##size vec, vec_##sign##int##bits out[size]) \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
129 { \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
130 _mm512_storeu_si512((__m512i *)out, ((union v##sign##int##bits##x##size##_impl_data *)&vec)->avx512f); \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
131 } \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
132 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
133 VEC_AVX512F_ADD_##bits##x##size(sign) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
134 VEC_AVX512F_SUB_##bits##x##size(sign) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
135 VEC_AVX512F_MUL_##bits##x##size(sign) \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
136 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
137 VEC_FUNC_IMPL v##sign##int##bits##x##size v##sign##int##bits##x##size##_avx512f_and(v##sign##int##bits##x##size vec1, v##sign##int##bits##x##size vec2) \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
138 { \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
139 union v##sign##int##bits##x##size##_impl_data *vec1d = (union v##sign##int##bits##x##size##_impl_data *)&vec1; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
140 union v##sign##int##bits##x##size##_impl_data *vec2d = (union v##sign##int##bits##x##size##_impl_data *)&vec2; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
141 \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
142 vec1d->avx512f = _mm512_and_si512(vec1d->avx512f, vec2d->avx512f); \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
143 return vec1d->vec; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
144 } \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
145 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
146 VEC_FUNC_IMPL v##sign##int##bits##x##size v##sign##int##bits##x##size##_avx512f_or(v##sign##int##bits##x##size vec1, v##sign##int##bits##x##size vec2) \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
147 { \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
148 union v##sign##int##bits##x##size##_impl_data *vec1d = (union v##sign##int##bits##x##size##_impl_data *)&vec1; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
149 union v##sign##int##bits##x##size##_impl_data *vec2d = (union v##sign##int##bits##x##size##_impl_data *)&vec2; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
150 \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
151 vec1d->avx512f = _mm512_or_si512(vec1d->avx512f, vec2d->avx512f); \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
152 return vec1d->vec; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
153 } \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
154 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
155 VEC_FUNC_IMPL v##sign##int##bits##x##size v##sign##int##bits##x##size##_avx512f_xor(v##sign##int##bits##x##size vec1, v##sign##int##bits##x##size vec2) \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
156 { \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
157 union v##sign##int##bits##x##size##_impl_data *vec1d = (union v##sign##int##bits##x##size##_impl_data *)&vec1; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
158 union v##sign##int##bits##x##size##_impl_data *vec2d = (union v##sign##int##bits##x##size##_impl_data *)&vec2; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
159 \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
160 vec1d->avx512f = _mm512_xor_si512(vec1d->avx512f, vec2d->avx512f); \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
161 return vec1d->vec; \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
162 } \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
163 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
164 VEC_AVX512F_LSHIFT_##bits##x##size(sign); \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
165 VEC_AVX512F_##sign##RSHIFT_##bits##x##size(sign); \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
166 VEC_AVX512F_LRSHIFT_##bits##x##size(sign); \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
167 \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
168 VEC_AVX512F_MINMAX_##sign##bits##x##size(min) \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
169 VEC_AVX512F_MINMAX_##sign##bits##x##size(max) \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
170 \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
171 const v##sign##int##bits##x##size##_impl v##sign##int##bits##x##size##_impl_avx512f = { \
28
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
172 .load_aligned = v##sign##int##bits##x##size##_avx512f_load_aligned, \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
173 .load = v##sign##int##bits##x##size##_avx512f_load, \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
174 .store_aligned = v##sign##int##bits##x##size##_avx512f_store_aligned, \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
175 .store = v##sign##int##bits##x##size##_avx512f_store, \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
176 .add = VEC_AVX512F_STRUCT_OPERATION_##bits##x##size(add, sign), \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
177 .sub = VEC_AVX512F_STRUCT_OPERATION_##bits##x##size(sub, sign), \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
178 .mul = VEC_AVX512F_STRUCT_OPERATION_##bits##x##size(mul, sign), \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
179 .band = v##sign##int##bits##x##size##_avx512f_and, \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
180 .bor = v##sign##int##bits##x##size##_avx512f_or, \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
181 .bxor = v##sign##int##bits##x##size##_avx512f_xor, \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
182 .lshift = VEC_AVX512F_STRUCT_OPERATION_##bits##x##size(lshift, sign), \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
183 .rshift = VEC_AVX512F_STRUCT_OPERATION_##bits##x##size(rshift, sign), \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
184 .lrshift = VEC_AVX512F_STRUCT_OPERATION_##bits##x##size(lrshift, sign), \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
185 .min = VEC_AVX512F_STRUCT_MINMAX_##bits##x##size(min, sign), \
c6c99ab1088a *: add min/max functions and a big big refactor (again)
Paper <paper@tflc.us>
parents: 24
diff changeset
186 .max = VEC_AVX512F_STRUCT_MINMAX_##bits##x##size(max, sign), \
23
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
187 };
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
188
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
189 #define VEC_AVX512F_DEFINE_OPERATIONS(bits, size) \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
190 VEC_AVX512F_DEFINE_OPERATIONS_SIGN(u, bits, size) \
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
191 VEC_AVX512F_DEFINE_OPERATIONS_SIGN( , bits, size)
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
192
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
193 VEC_AVX512F_DEFINE_OPERATIONS(32, 16)
e26874655738 *: huge refactor, new major release (hahaha)
Paper <paper@tflc.us>
parents:
diff changeset
194 VEC_AVX512F_DEFINE_OPERATIONS(64, 8)