Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

For a 0.6.0 release #277

Merged
merged 67 commits into from
Sep 29, 2024
Merged
Changes from 5 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
abfabb5
readme typo
ablaom Jun 16, 2024
07e01b7
readme typos
ablaom Jun 16, 2024
70dff6e
Update README.md
EssamWisam Jul 3, 2024
fb98c27
✅ Add CategoricalEmbedding layer with documentation and tests
EssamWisam Aug 3, 2024
3e67625
➕ Add ordinal encoder
EssamWisam Aug 4, 2024
d77d20d
fix MultitargetNeuralNetworkRegressor doctring to close #268
ablaom Aug 4, 2024
d2fc2a9
➕ Add embedding transformer and more test encoder tests
EssamWisam Aug 5, 2024
41ca8e0
𝚫 Override fitresult and input scitypes for the four models
EssamWisam Aug 5, 2024
afab1e0
𝚫 Change name of embedding layer
EssamWisam Aug 5, 2024
86fa74c
👨‍💻 Refactor fit and update in mlj_model_interface
EssamWisam Aug 5, 2024
33cb7d9
🔥 Delete replaced file
EssamWisam Aug 5, 2024
cf30f90
➕ Include necessary files in MLJFlux.jl
EssamWisam Aug 5, 2024
48b1ce2
✅ Expose embedding_dims hyperparameter and update docs
EssamWisam Aug 5, 2024
d1e65c0
✅ Finish adding entity embedding tests
EssamWisam Aug 5, 2024
af91ee9
👨‍🔧 Fix small error
EssamWisam Aug 5, 2024
e7b0fff
🔥 Delete old file
EssamWisam Aug 6, 2024
833b845
🤔 Make all input tables float
EssamWisam Aug 6, 2024
962bbda
🤔 Further ensure all inputs are float
EssamWisam Aug 6, 2024
1e41256
Merge pull request #270 from FluxML/multitarget-docstring-fix
ablaom Aug 6, 2024
4ff7cbe
Update src/types.jl
EssamWisam Aug 7, 2024
878a905
Update src/mlj_model_interface.jl
EssamWisam Aug 7, 2024
c90cf6e
Update src/MLJFlux.jl
EssamWisam Aug 31, 2024
4547f2a
Update src/classifier.jl
EssamWisam Aug 31, 2024
e8e87cd
Update src/regressor.jl
EssamWisam Aug 31, 2024
9a836b2
Update src/regressor.jl
EssamWisam Aug 31, 2024
6fb81ff
✅ Fix type for binary classifier
EssamWisam Aug 31, 2024
3a8bdcb
➕ No longer export EntityEmbedder
EssamWisam Sep 1, 2024
f7abf07
✍🏻 Improve docstring
EssamWisam Sep 1, 2024
bbe121f
🚨 Get rid of ScientificTypes
EssamWisam Sep 1, 2024
706b630
Follow up
EssamWisam Sep 1, 2024
b0d8e44
🔼 Better docstrings
EssamWisam Sep 1, 2024
4458627
✅ Add missing dependency
EssamWisam Sep 1, 2024
f1f7dfe
✅ Improve docstring
EssamWisam Sep 1, 2024
daa4b2a
✅ Better default for embedding dims
EssamWisam Sep 1, 2024
59c9bb9
➕ Include in top-level only
EssamWisam Sep 1, 2024
749a247
👨‍🔧 Fix docstring spacing
EssamWisam Sep 1, 2024
46577f7
✅ Enforce columns
EssamWisam Sep 1, 2024
e5d8141
✅ Add trait for embedding model
EssamWisam Sep 1, 2024
f60f789
✅ Get rid of case distinction
EssamWisam Sep 1, 2024
5fd85a3
🌟 Introduce integration tests
EssamWisam Sep 1, 2024
2cc227a
Add `Tables.columns`
EssamWisam Sep 1, 2024
eac7727
✅ Bring back is_entity_enabled and more
EssamWisam Sep 1, 2024
cc56439
✅ Raise tolerance
EssamWisam Sep 1, 2024
b1a90b2
automatically convert input matrix to Float32
tiemvanderdeure Sep 6, 2024
407edc5
🫧 Some final polishing
EssamWisam Sep 8, 2024
21a2fdd
tweak document strings
ablaom Sep 9, 2024
e0a2eb8
tag some methods as private to avoid any possible ambiguity
ablaom Sep 9, 2024
310cb12
bump 0.6.0
ablaom Sep 9, 2024
945016d
Merge pull request #267 from FluxML/entity-embeddings
ablaom Sep 9, 2024
ff32ca5
dispatch on array not matrix
tiemvanderdeure Sep 9, 2024
56838bf
pass verbosity to collate in update
tiemvanderdeure Sep 9, 2024
5ae2fe5
add tests and fix existing ones
tiemvanderdeure Sep 9, 2024
7818959
✅ Make embedding model output Float32
EssamWisam Sep 14, 2024
493d23b
Fix ordinal encoding float types
EssamWisam Sep 16, 2024
eac2974
Merge remote-tracking branch 'origin/entity-embeddings' into double-t…
ablaom Sep 16, 2024
1cfcd1e
Merge remote-tracking branch 'tiemvanderdeure/convert_to_f32' into do…
ablaom Sep 16, 2024
09b17da
fix some tests
ablaom Sep 16, 2024
c57870b
Fix ordinal encoding ransform type
EssamWisam Sep 26, 2024
804c0c5
Merge remote-tracking branch 'origin/entity-embeddings' into double-t…
ablaom Sep 26, 2024
87c8f66
ensure ordinal encoder returns concrete type
ablaom Sep 26, 2024
1adf9cf
tweaks, and relax one tolerance
ablaom Sep 26, 2024
9aeb7ba
try reducing tolerance again
ablaom Sep 26, 2024
fcc4cf8
reduce again
ablaom Sep 26, 2024
3bf0902
and again
ablaom Sep 26, 2024
b2bafd2
✅ Fix float type
EssamWisam Sep 28, 2024
bc805e4
remove v -> [v...] from output of encoder
ablaom Sep 29, 2024
19d4275
Merge pull request #276 from FluxML/double-trouble
ablaom Sep 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -33,6 +33,7 @@ julia = "1.9"
[extras]
CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
MLJBase = "a7f614a8-145f-11e9-1d2a-a57a1082229d"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
StableRNGs = "860ef19b-820b-49d6-a774-d7a799459cd3"
@@ -42,4 +43,4 @@ Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
cuDNN = "02a925ec-e4fe-4b08-9a7e-0d78e3d38ccd"

[targets]
test = ["CUDA", "cuDNN", "LinearAlgebra", "MLJBase", "Random", "StableRNGs", "StatisticalMeasures", "StatsBase", "Test"]
test = ["CUDA", "cuDNN", "LinearAlgebra", "Logging", "MLJBase", "Random", "StableRNGs", "StatisticalMeasures", "StatsBase", "Test"]
15 changes: 11 additions & 4 deletions src/core.jl
Original file line number Diff line number Diff line change
@@ -276,15 +276,22 @@ input `X` and target `y` in the form required by
by `model.batch_size`.)

"""
function collate(model, X, y)
function collate(model, X, y, verbosity)
row_batches = Base.Iterators.partition(1:nrows(y), model.batch_size)
Xmatrix = reformat(X)
Xmatrix = _f32(reformat(X), verbosity)
ymatrix = reformat(y)
return [_get(Xmatrix, b) for b in row_batches], [_get(ymatrix, b) for b in row_batches]
end
function collate(model::NeuralNetworkBinaryClassifier, X, y)
function collate(model::NeuralNetworkBinaryClassifier, X, y, verbosity)
row_batches = Base.Iterators.partition(1:nrows(y), model.batch_size)
Xmatrix = reformat(X)
Xmatrix = _f32(reformat(X), verbosity)
yvec = (y .== classes(y)[2])' # convert to boolean
return [_get(Xmatrix, b) for b in row_batches], [_get(yvec, b) for b in row_batches]
end

_f32(x::AbstractArray{Float32}, verbosity) = x
function _f32(x::AbstractArray, verbosity)
verbosity > 0 && @info "MLJFlux: converting input data to Float32"
return Float32.(x)
end

10 changes: 5 additions & 5 deletions src/mlj_model_interface.jl
Original file line number Diff line number Diff line change
@@ -84,7 +84,7 @@ function MLJModelInterface.fit(model::MLJFluxModel,
featnames = Tables.schema(X).names
end

# entityprops is (index = cat_inds[i], levels = num_levels[i], newdim = newdims[i])
# entityprops is (index = cat_inds[i], levels = num_levels[i], newdim = newdims[i])
# for each categorical feature
default_embedding_dims = enable_entity_embs ? model.embedding_dims : Dict{Symbol, Real}()
entityprops, entityemb_output_dim =
@@ -103,8 +103,8 @@ function MLJModelInterface.fit(model::MLJFluxModel,
entityemb_output_dim,
)

# Format data as needed by Flux and move to GPU
data = move.(collate(model, X, y))
# Format data as needed by Flux and move to GPU
data = move.(collate(model, X, y, verbosity))

# Test chain works (as it may be custom)
x = data[1][1]
@@ -143,7 +143,7 @@ function MLJModelInterface.fit(model::MLJFluxModel,
featnames,
)

# Prepare fitresult
# Prepare fitresult
fitresult =
MLJFlux.fitresult(model, Flux.cpu(chain), y, ordinal_mappings, embedding_matrices)

@@ -216,7 +216,7 @@ function MLJModelInterface.update(model::MLJFluxModel,
chain = construct_model_chain(model, rng, shape, move)
end
# reset `optimiser_state`:
data = move.(collate(model, X, y))
data = move.(collate(model, X, y, verbosity))
regularized_optimiser, optimiser_state =
prepare_optimiser(data, model, chain)
epochs = model.epochs
19 changes: 12 additions & 7 deletions test/core.jl
Original file line number Diff line number Diff line change
@@ -14,23 +14,28 @@ rowvec(y::Vector) = reshape(y, 1, length(y))
end

@testset "collate" begin
# NeuralNetworRegressor:
Xmatrix = broadcast(x->round(x, sigdigits=2), rand(stable_rng, 10, 3))
Xmatrix = broadcast(x->round(x, sigdigits=2), rand(stable_rng, Float32, 10, 3))
Xmat_f64 = Float64.(Xmatrix)
# convert to a column table:
X = MLJBase.table(Xmatrix)
X_64 = MLJBase.table(Xmat_f64)

# NeuralNetworRegressor:
y = rand(stable_rng, Float32, 10)
model = MLJFlux.NeuralNetworkRegressor()
model.batch_size= 3
@test MLJFlux.collate(model, X, y) ==
@test MLJFlux.collate(model, X, y, 1) == MLJFlux.collate(model, X_64, y, 1) ==
([Xmatrix'[:,1:3], Xmatrix'[:,4:6], Xmatrix'[:,7:9], Xmatrix'[:,10:10]],
rowvec.([y[1:3], y[4:6], y[7:9], y[10:10]]))
@test_logs (:info,) MLJFlux.collate(model, X_64, y, 1)
@test_logs min_level=Logging.Info MLJFlux.collate(model, X, y, 1)
@test_logs min_level=Logging.Info MLJFlux.collate(model, X, y, 0)

# NeuralNetworClassifier:
y = categorical(['a', 'b', 'a', 'a', 'b', 'a', 'a', 'a', 'b', 'a'])
model = MLJFlux.NeuralNetworkClassifier()
model.batch_size = 3
data = MLJFlux.collate(model, X, y)
data = MLJFlux.collate(model, X, y, 1)

@test data == ([Xmatrix'[:,1:3], Xmatrix'[:,4:6],
Xmatrix'[:,7:9], Xmatrix'[:,10:10]],
@@ -42,13 +47,13 @@ end
y = MLJBase.table(ymatrix) # a rowaccess table
model = MLJFlux.NeuralNetworkRegressor()
model.batch_size= 3
@test MLJFlux.collate(model, X, y) ==
@test MLJFlux.collate(model, X, y, 1) ==
([Xmatrix'[:,1:3], Xmatrix'[:,4:6], Xmatrix'[:,7:9], Xmatrix'[:,10:10]],
rowvec.([ymatrix'[:,1:3], ymatrix'[:,4:6], ymatrix'[:,7:9],
ymatrix'[:,10:10]]))

y = Tables.columntable(y) # try a columnaccess table
@test MLJFlux.collate(model, X, y) ==
@test MLJFlux.collate(model, X, y, 1) ==
([Xmatrix'[:,1:3], Xmatrix'[:,4:6], Xmatrix'[:,7:9], Xmatrix'[:,10:10]],
rowvec.([ymatrix'[:,1:3], ymatrix'[:,4:6],
ymatrix'[:,7:9], ymatrix'[:,10:10]]))
@@ -58,7 +63,7 @@ end
y = categorical(['a', 'b', 'a', 'a', 'b', 'a', 'a', 'a', 'b', 'a'])
model = MLJFlux.ImageClassifier(batch_size=2)

data = MLJFlux.collate(model, Xmatrix, y)
data = MLJFlux.collate(model, Xmatrix, y, 1)
@test first.(data) == (Float32.(cat(Xmatrix[1], Xmatrix[2], dims=4)),
rowvec.([1 0;0 1]))

2 changes: 1 addition & 1 deletion test/mlj_model_interface.jl
Original file line number Diff line number Diff line change
@@ -94,7 +94,7 @@ end
# (2) manually train for one epoch explicitly adding a loss penalty:
chain = MLJFlux.build(builder, StableRNG(123), 3, 1);
penalty = Penalizer(lambda, alpha); # defined in test_utils.jl
X, y = MLJFlux.collate(model, Xuser, yuser);
X, y = MLJFlux.collate(model, Xuser, yuser, 0);
loss = model.loss;
n_batches = div(nobservations, batch_size)
optimiser_state = Optimisers.setup(optimiser, chain);
1 change: 1 addition & 0 deletions test/runtests.jl
Original file line number Diff line number Diff line change
@@ -14,6 +14,7 @@ using StableRNGs
using CUDA, cuDNN
import StatisticalMeasures
import Optimisers
import Logging

using ComputationalResources
using ComputationalResources: CPU1, CUDALibs