Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

My code has an issue identifying the search space in grid search. #5417

Closed
jobmart opened this issue Dec 27, 2024 · 2 comments
Closed

My code has an issue identifying the search space in grid search. #5417

jobmart opened this issue Dec 27, 2024 · 2 comments
Labels
invalid/unrelated unrelated to this project or invalid type of issues needs-more-info More info is needed to complete the issue

Comments

@jobmart
Copy link

jobmart commented Dec 27, 2024

"I am working on the backend code of my Jupyter Notebook, and I'm encountering an issue when running the model. Here's the code:
`# Imports
import numpy as np
import pandas as pd
from scipy import stats

Sklearn

from sklearn.metrics import r2_score
from ML.ml_utils import *
from sklearn.model_selection import train_test_split

FNN

import tensorflow as tf
import keras_tuner as kt
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, BatchNormalization
from tensorflow.keras.optimizers import SGD, Adam
from tensorflow.keras.callbacks import EarlyStopping
from keras_tuner import HyperParameters
from keras_tuner import Objective
from keras_tuner.tuners import GridSearch, RandomSearch
from your_module import create_model

FeedForwardNN and related classes

class FeedForwardNN(tf.keras.Model):
def init(self, input_dim=None, random_seed=42):
super(FeedForwardNN, self).init()
self.seed = random_seed
self.input_dim = 2048
self.model = self.build_model()

def build_model(self, hp=None):
    if hp is None:
        hp = kt.HyperParameters()
    tf.random.set_seed(self.seed)

    # Define hyperparameters
    optimizer_c = hp.Choice("optimizer", ['SGD', 'Adam'])
    learning_rate = hp.Float("learning_rate", min_value=0.00001, max_value=0.1, sampling='log')
    l2_reg = hp.Float("l2_reg", min_value=0.0001, max_value=0.1, sampling='log')
    dropout_rate = hp.Float("dropout_rate", min_value=0, max_value=0.5, step=0.1)
    output_dim = hp.Fixed("output_dim", value=1)

    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Input(shape=(self.input_dim,)))
    model.add(tf.keras.layers.Dense(units=hp.Int("units_1", min_value=32, max_value=256, step=32),
                                    activation='relu',
                                    kernel_regularizer=tf.keras.regularizers.l2(l2_reg)))

    for i in range(hp.Int("num_layers", 1, 3)):
        model.add(tf.keras.layers.Dense(units=hp.Int(f"units_{i}", min_value=32, max_value=256, step=32),
                                        activation='relu',
                                        kernel_regularizer=tf.keras.regularizers.l2(l2_reg)))

    if optimizer_c == "SGD":
        optimizer = tf.keras.optimizers.SGD(learning_rate=learning_rate)
    elif optimizer_c == "Adam":
        optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)
    else:
        raise ValueError("Unsupported optimizer")

    model.add(tf.keras.layers.Dropout(dropout_rate))
    model.add(tf.keras.layers.Dense(output_dim, activation='linear'))

    model.compile(loss=tf.keras.losses.mean_squared_error,
                  optimizer=optimizer,
                  metrics=['mean_absolute_error'])
    return model

def call(self, inputs, training=False):
    return self.model(inputs, training=training)

def fit(self, X, y, *args, **kwargs):
    return self.model.fit(X, y, *args, shuffle=True, **kwargs)

When I attempt to run the hyperparameter search with this setup, the search_space_summary() displays the following output:Search space summary
Default search space size: 0
`
As a result, the hyperparameter search doesn't execute. I suspect the issue is related to the hp=kt.HyperParameters configuration, but I have tried multiple approaches without success.

Does anyone have suggestions for how to correct this and enable proper hyperparameter tuning?"

Copy link

You've chosen to report an unexpected problem or bug. Unless you already know the root cause of it, please include details about it by filling the issue template.
The following information is missing: "Instructions To Reproduce the Issue and Full Logs"; "Your Environment";

@github-actions github-actions bot added the needs-more-info More info is needed to complete the issue label Dec 27, 2024
Copy link

github-actions bot commented Jan 4, 2025

Requested information was not provided in 7 days, so we're closing this issue.

Please open new issue if information becomes available. Otherwise, use github discussions for free-form discussions.

@github-actions github-actions bot closed this as completed Jan 4, 2025
@github-actions github-actions bot added the invalid/unrelated unrelated to this project or invalid type of issues label Jan 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid/unrelated unrelated to this project or invalid type of issues needs-more-info More info is needed to complete the issue
Projects
None yet
Development

No branches or pull requests

1 participant