Error during Running the training

[6monkeyrs]

I followed the training steps and I am re-using your images. I get the following error and was wondering if anyone has been able to figure out the issue:

(tensorflow1) C:\tensorflow1\models\research\object_detection>python train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/faster_rcnn_inception_v2_pets.config
WARNING:tensorflow:From C:\tensorflow1\models\research\object_detection\trainer.py:260: create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.create_global_step
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
INFO:tensorflow:Scale of 0 disables regularizer.
INFO:tensorflow:Scale of 0 disables regularizer.
INFO:tensorflow:depth of additional conv before box predictor: 0
WARNING:tensorflow:From C:\tensorflow1\models\research\object_detection\core\box_predictor.py:407: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
INFO:tensorflow:Scale of 0 disables regularizer.
WARNING:tensorflow:From C:\tensorflow1\models\research\object_detection\core\losses.py:317: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See @{tf.nn.softmax_cross_entropy_with_logits_v2}.

Traceback (most recent call last):
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\ops.py", line 1104, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\constant_op.py", line 235, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\constant_op.py", line 214, in constant
value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 441, in make_tensor_proto
_GetDenseDimensions(values)))
ValueError: Argument must be a dense tensor: range(0, 3) - got shape [3], but wanted [].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 524, in _apply_op_helper
values, as_ref=input_arg.is_ref).dtype.name
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\ops.py", line 1104, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\constant_op.py", line 235, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\constant_op.py", line 214, in constant
value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 441, in make_tensor_proto
_GetDenseDimensions(values)))
ValueError: Argument must be a dense tensor: range(0, 3) - got shape [3], but wanted [].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "train.py", line 184, in
tf.app.run()
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\platform\app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 180, in main
graph_hook_fn=graph_rewriter_fn)
File "C:\tensorflow1\models\research\object_detection\trainer.py", line 298, in train
train_config.optimizer)
File "C:\tensorflow1\models\research\object_detection\builders\optimizer_builder.py", line 50, in build
learning_rate = _create_learning_rate(config.learning_rate)
File "C:\tensorflow1\models\research\object_detection\builders\optimizer_builder.py", line 109, in _create_learning_rate
learning_rate_sequence, config.warmup)
File "C:\tensorflow1\models\research\object_detection\utils\learning_schedules.py", line 169, in manual_stepping
[0] * num_boundaries))
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\ops\array_ops.py", line 2681, in where
return gen_math_ops.select(condition=condition, x=x, y=y, name=name)
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 7189, in select
"Select", condition=condition, t=x, e=y, name=name)
File "C:\Users\ferna\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 528, in _apply_op_helper
(input_name, err))
ValueError: Tried to convert 't' to a tensor and failed. Error: Argument must be a dense tensor: range(0, 3) - got shape [3], but wanted [].

[vinceyang]

modify models/research/object_detection/utils/learning_schedules.py lines 167-169.
rate_index = tf.reduce_max(tf.where(tf.greater_equal(global_step, boundaries), range(num_boundaries), [0] * num_boundaries))
to
rate_index = tf.reduce_max(tf.where(tf.greater_equal(global_step, boundaries), list(range(num_boundaries)), [0] * num_boundaries))

and reinstall object_detection model:

`\models\research> python setup.py build

\models\research> python setup.py install`

you can refer:
tensorflow/models#3705 (comment)

[6monkeyrs]

Thanks, this did resolved my issue.

[shABanty]

I also got problem while training.

(tensorflow_cpu) C:\Users\AB\Tensorflow\workplace\training_demo>python train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/ssd_inception_v2_coco.config
WARNING:tensorflow:From C:\Users\AB\Anaconda3\envs\tensorflow_cpu\lib\site-packages\tensorflow\python\platform\app.py:125: main (from main) is deprecated and will be removed in a future version.
Instructions for updating:
Use object_detection/model_main.py.
WARNING:tensorflow:From C:\Users\AB\Anaconda3\envs\tensorflow_cpu\lib\site-packages\object_detection\legacy\trainer.py:266: create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.create_global_step
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
WARNING:tensorflow:From C:\Users\AB\Anaconda3\envs\tensorflow_cpu\lib\site-packages\object_detection\core\preprocessor.py:1240: calling squeeze (from tensorflow.python.ops.array_ops) with squeeze_dims is deprecated and will be removed in a future version.
Instructions for updating:
Use the axis argument instead
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
WARNING:tensorflow:From C:\Users\AB\Anaconda3\envs\tensorflow_cpu\lib\site-packages\tensorflow\contrib\slim\python\slim\learning.py:737: Supervisor.init (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
2020-02-07 01:57:41.478395: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
INFO:tensorflow:Restoring parameters from C:\Users\AB\Tensorflow\workplace\training_demo\pre-trained-model\model.ckpt
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Saving checkpoint to path training/model.ckpt
INFO:tensorflow:Starting Queues.
INFO:tensorflow:global_step/sec: 0
2020-02-07 01:58:16.645162: I T:\src\github\tensorflow\tensorflow\core\kernels\data\shuffle_dataset_op.cc:95] Filling up shuffle buffer (this may take a while): 480 of 2048
2020-02-07 01:58:26.795941: I T:\src\github\tensorflow\tensorflow\core\kernels\data\shuffle_dataset_op.cc:95] Filling up shuffle buffer (this may take a while): 633 of 2048
2020-02-07 0

Any solutions? Pls help!! Thx