### Author:  [@ElyMatos](https://forum.thousandbrains.org/u/ElyMatos)
#### Post date: [March 12, 2025, 4:28pm UTC](https://forum.thousandbrains.org/t/error-trying-to-create-an-omniglot-experiment/489/1 "2025-03-12T16:28:38Z")
Hi!
I’m trying to create an experiment to use Omniglot dataset. I tried just to copy from the first experiment from tutorial, and change what appeared to be necessary. I got:
```auto
two_d_experiment = dict(
experiment_class=MontyExperiment,
logging_config=LoggingConfig(),
experiment_args=ExperimentArgs(
do_eval=False,
max_train_steps=1,
n_train_epochs=1,
),
monty_config=SingleCameraMontyConfig(),
# Data{set, loader} config
dataset_class=OmniglotEnvironment,
dataset_args=OmniglotDatasetArgs(),
train_dataloader_class=OmniglotDataLoader,
train_dataloader_args=OmniglotDataloaderArgs(),
eval_dataloader_class=OmniglotDataLoader,
eval_dataloader_args=OmniglotDataloaderArgs(),
)
```
(I’m sure that it is not so simple 🙂 anyway it was worth to try…
But I’m getting following errors:
```auto
run.py -e two_d_experiment
MONTY_LOGS not set. Using default directory: ~/tbp/results/monty/
MONTY_MODELS not set. Using default directory: ~/tbp/results/monty/pretrained_models/
MONTY_DATA not set. Using default directory: /home/ematos/tbp/data/
WANDB_DIR not set. Using default directory: ~/tbp/results/monty/
Traceback (most recent call last):
File "/home/ematos/tbp/src/tbp/monty/frameworks/run.py", line 48, in run
exp.setup_experiment(config)
File "/home/ematos/tbp/src/tbp/monty/frameworks/experiments/monty_experiment.py", line 71, in setup_experiment
self.load_dataset_and_dataloaders(config)
File "/home/ematos/tbp/src/tbp/monty/frameworks/experiments/monty_experiment.py", line 180, in load_dataset_and_dataloaders
self.dataset = self.load_dataset(dataset_class, dataset_args)
File "/home/ematos/tbp/src/tbp/monty/frameworks/experiments/monty_experiment.py", line 220, in load_dataset
raise TypeError("dataset class must be EnvironmentDataset (for now)")
TypeError: dataset class must be EnvironmentDataset (for now)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ematos/tbp/benchmarks/run.py", line 44, in
main(all_configs=CONFIGS, experiments=cmd_args.experiments)
File "/home/ematos/tbp/src/tbp/monty/frameworks/run.py", line 115, in main
run(exp_config)
File "/home/ematos/tbp/src/tbp/monty/frameworks/run.py", line 59, in run
exp.evaluate()
File "/home/ematos/tbp/src/tbp/monty/frameworks/experiments/monty_experiment.py", line 628, in __exit__
self.close()
File "/home/ematos/tbp/src/tbp/monty/frameworks/experiments/monty_experiment.py", line 596, in close
if isinstance(self.dataset, EnvironmentDataset):
AttributeError: 'MontyExperiment' object has no attribute 'dataset'
Process finished with exit code 1
```
Because the message
`TypeError: dataset class must be EnvironmentDataset (for now)`
I understood that class `OmniglotEnvironment` is not the correct to be used…
Any help is appreciated 🙂
Thanks!
---
### Author:  [@vclay](https://forum.thousandbrains.org/u/vclay)
#### Post date: [March 13, 2025, 6:26am UTC](https://forum.thousandbrains.org/t/error-trying-to-create-an-omniglot-experiment/489/2 "2025-03-13T06:26:45Z")
Hi Ely,
sorry about the trouble with this. I hope the code still works since we haven’t run experiments with the Omniglot dataset in a while (we figured that we need to be able to model compositional objects before we can get good accuracy on this dataset).
For an example config that we used in the past, you could have a look at this one: [monty\_lab/experiments/configs/graph\_experiments.py at f8fe7290eef7b198f01f8fd22c64a5585f495ee1 · thousandbrainsproject/monty\_lab · GitHub](https://github.com/thousandbrainsproject/monty_lab/blob/f8fe7290eef7b198f01f8fd22c64a5585f495ee1/experiments/configs/graph_experiments.py#L778)
There we just use `dataset_class=ED.EnvironmentDataset`
The `OmniglotEnvironment` is specified in `env_init_func: Callable = field(default=OmniglotEnvironment)` in the `OmniglotDatasetArgs` class.
Hope this helps!
-Viviane
---
### Author:  [@ElyMatos](https://forum.thousandbrains.org/u/ElyMatos)
#### Post date: [March 13, 2025, 12:22pm UTC](https://forum.thousandbrains.org/t/error-trying-to-create-an-omniglot-experiment/489/3 "2025-03-13T12:22:42Z")
Thanks, @vclay 🙂
I was able to run the experiment using that configuration and after make two small changes in the current code:
```auto
class HabitatDistantPatchSM(DetailedLoggingSM, NoiseMixin):
...
def __init__ (
motor_only_step=False,
):
...
self.motor_only_step = motor_only_step
```
and
```auto
class OmniglotEnvironment(EmbodiedEnvironment):
...
def step(self, _action, amount = 1):
```
but I’m not sure if it works at the end, as I got no matching:
(I don’t know if this is the expected result…).
About modeling compositional objects, is this a working in progress or we already have something to test with Omniglot dataset?
Thanks!
---
### Author:  [@vclay](https://forum.thousandbrains.org/u/vclay)
#### Post date: [March 14, 2025, 5:35pm UTC](https://forum.thousandbrains.org/t/error-trying-to-create-an-omniglot-experiment/489/4 "2025-03-14T17:35:43Z")
Hi Ely,
if you are using exactly the same config, then you should be able to recognize these digits I think. Are you loading pretrained models like here: [monty\_lab/experiments/configs/graph\_experiments.py at f8fe7290eef7b198f01f8fd22c64a5585f495ee1 · thousandbrainsproject/monty\_lab · GitHub](https://github.com/thousandbrainsproject/monty_lab/blob/f8fe7290eef7b198f01f8fd22c64a5585f495ee1/experiments/configs/graph_experiments.py#L784)
To generate these models you will have to run a pertaining experiment like this first: [monty\_lab/experiments/configs/more\_pretraining\_experiments.py at f8fe7290eef7b198f01f8fd22c64a5585f495ee1 · thousandbrainsproject/monty\_lab · GitHub](https://github.com/thousandbrainsproject/monty_lab/blob/f8fe7290eef7b198f01f8fd22c64a5585f495ee1/experiments/configs/more_pretraining_experiments.py#L580)
Otherwise the learning module has nothing in its memory and can therefor also not recognize anything. Since all or your episodes have no\_match with result = new\_object0, it looks like you are in evaluation mode (no new information being added to memory) and you have nothing in memory.
Compositional objects are the next big milestone on our [roadmap](https://docs.google.com/spreadsheets/d/1HlKqhMzYWxpW6_D0LtDd5qEuWRYHl6xMfxbIZ76lzng/edit?usp=sharing) (see row 50). We already made a good amount of progress on this (figuring out how it conceptually works in the brain and how we want to implement it + adding Monty support for stacking LMs hierarchically) but there are still a few remaining items left for it to work well (listed in that spreadsheet).
Best wishes,
Viviane
---
### Author:  [@vclay](https://forum.thousandbrains.org/u/vclay)
#### Post date: [May 5, 2025, 1:02pm UTC](https://forum.thousandbrains.org/t/error-trying-to-create-an-omniglot-experiment/489/5 "2025-05-05T13:02:36Z")
Hi @ElyMatos
I just wanted to mention that I added two new tutorials to our documentation ([New Tutorials on Using Monty in Custom Applications](https://forum.thousandbrains.org/t/new-tutorials-on-using-monty-in-custom-applications/588)) One of them [talks through the Omniglot example in detail](https://thousandbrainsproject.readme.io/docs/using-monty-in-a-custom-application#example-1-omniglot) and also provides code to follow along.
As I was writing it I actually noticed that the OmniglotEnvironment code was broken and added a [fix](https://github.com/thousandbrainsproject/tbp.monty/pull/245). Maybe give it a try again now! Let me know if you run into any issues.
- Viviane