결론

• 서브쿼리와 랜덤 값을 이용한 MySQL 쿼리 방법을 이용하여 데이터 삽입 시간을 22.42분으로 단축 성공
• Spring Boot Test를 이용한 방법보다 약 515.63분 더 빠른 결과 ( 95.78% 빠른 속도를 보여줌)

1. Spring Boot Test를 이용

• 접근이 쉬워서 처음 시도한 방식
• 50,000건(stream) 시도
• 처리 결과 : 538분 소요

사용 코드

@SpringBootTest
public class GenerateTest {
	@Autowired
	private StreamRepository streamsRepository;
	
	@Autowired
	private AdViewRepository adViewsRepository;
	
	@Autowired
	private AdRepository adsRepository;
	
	@Autowired
	private VideoRepository videosRepository;
	
	@Autowired
	private UserRepository userRepository;
	
	@Autowired
	private VideoAdRepository videoAdRepository;
	
	public void generateData(int numRecords) {
	    Random random = new Random();
	    Users user = userRepository.findById(1L).orElseThrow(() -> new IllegalArgumentException("User not found"));
	    List<Videos> videos = videosRepository.findAll();
	    List<Ads> ads = adsRepository.findAll();
	    List<VideoAd> videoAds = videoAdRepository.findAll();
	
	    for (Videos video : videos) {
	        for (int i = 0; i < numRecords; i++) {
	            int playTime = random.nextInt(1800) + 300;  // 300초에서 1800초 사이의 랜덤 시간
	            Streams stream = new Streams();
	            stream.setCreatedAt(LocalDateTime.now().minusDays(1));
	            stream.setPlayTime(playTime);
	            stream.setUser(user);
	            stream.setVideo(video);
	            streamsRepository.save(stream);
	
	            int numAds = playTime / 300;
	
	            for (int j = 0; j < numAds; j++) {
	                VideoAd videoAd = videoAds.get(random.nextInt(videoAds.size()));
	                AdViews adView = new AdViews();
	                adView.setCreatedAt(LocalDate.now().minusDays(1));
	                adView.setVideoAd(videoAd);
	                adViewsRepository.save(adView);
	            }
	        }
	    }
	}
	
	@Test
	public void testGenerateData() { 
	    generateData(10000);  // 비디오 개수*10000개 생성
	}
}

삽입 완료 결과

OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
> Task :test
2024-07-23T11:28:15.058+09:00  INFO 7744 --- [binplay] [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2024-07-23T11:28:15.064+09:00  INFO 7744 --- [binplay] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2024-07-23T11:28:15.086+09:00  INFO 7744 --- [binplay] [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.
BUILD SUCCESSFUL in 26m 55s
4 actionable tasks: 2 executed, 2 up-to-date
AM 11:28:18: Execution finished ':test --tests "com.sparta.binplay.GenerateTest"'.

⚠️ 100,000건 한번에 넣다가 다운됨

2. 프로시저 사용

• 배치 처리를 통해 대량의 데이터를 효율적으로 삽입