ERROR: Too many ROS containers exist for the following projections:

it caused by your batch size too small, which lead to a lot of ROS Containers created and reach the limitation(1024 default). you should do defragment using TupleMover task(mergeout) before the error raised.

To do troubleshooting:

  1. ROS Containers viewed from the projections.
select * from STORAGE_CONTAINERS where projection_name like '%DATASET_TABLE%';
  1. check ContainersPerProjectionLimit settings Views
WHERE parameter_name = 'ContainersPerProjectionLimit' ;
  1. ROS Container Number Lookup
select count(*) from STORAGE_CONTAINERS where projection_name like '%DATASET_TABLE%';
  1. Solutions
-- change ContainersPerProjectionLimit settings
SELECT SET_CONFIG_PARAMETER('ContainersPerProjectionLimit', 2048);


-- change mergeout frequency


-- Do TupleMover Task manually
select do_tm_task('mergeout','projection_name')

COPY Rejected Data and Exception Files

it often happened some rows with format issue, data type issue rejected by copy command while try load it by copy command. the query return succeed but some of data rejected.

To do troubleshooting

  1. save Rejected Data and Exceptions
COPY large_tbl FROM :file1 ON site01,
               :file2 ON site01,
               :file3 ON site02,
               :file4 ON site02
               DELIMITER '|'
               REJECTED DATA :reject_s1 ON site01, :reject_s2 ON site02 
               EXCEPTIONS :except_s1 ON site01, :except_s2 ON site02;
  1. check errors in exceptions file, and fix it one by one