Crystal: Module validation failed: Stored value type does not match pointer operand type!

Created on 5 Jun 2017 · 11Comments · Source: crystal-lang/crystal

Just came across this, this is my first bug, so not sure if I've done this right, and couldn't see any duplicates. If you need any more information just ask, more than happy to provide. Not sure which part of my source to include with regards to this, and cannot include the whole project as it's a company project.

Crystal Version:
Crystal 0.22.0 (2017-04-20) LLVM 4.0.0

Error:
```Module validation failed: Stored value type does not match pointer operand type!
store %Nil zeroinitializer, i32* %127, !dbg !374
i32
0x10a64f063: *raise:NoReturn at ??
0x10b62f3ab: *Crystal::CodeGenVisitor#finish:Nil at ??
0x10b55a97e: *Crystal::Compiler#codegen:(Tuple(Array(Crystal::Compiler::CompilationUnit), Array(String)) | Nil) at ??
0x10bdff094: *Crystal::Compiler#compile:Crystal::Compiler::Result at ??
0x10a6bc40b: *Crystal::Command#run:(Bool | Crystal::Compiler::Result | IO::FileDescriptor | Nil) at ??
0x10a6884bd: main at ??

Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues

Issue seems to stem from these two functions:

DB Function:

def db_proc(query_string : String, alternate_db=nil, exec=:single_result)
if @debug == true
puts "Alternate DB to Use: ".colorize(:yellow).to_s + alternate_db.colorize(:blue).to_s unless alternate_db.nil?
puts "No alternate DB provided, using default #{@scheduler_db[:database]}".colorize(:yellow) if alternate_db.nil?
puts query_string.colorize(:yellow)
end
case exec
when :single_result
DB.open "#{@scheduler_db[:type]}://#{@scheduler_db[:username]}:#{@scheduler_db[:password]}@#{@scheduler_db[:ip]}:#{@scheduler_db[:port]}/#{alternate_db.nil? ? @scheduler_db[:database] : alternate_db}" do |db|
db.query query_string do |results|
# Check if multiple rows or just one result
while results.move_next
yield(results) unless results.nil?
end
end
end
when :multi_result
DB.open "#{@scheduler_db[:type]}://#{@scheduler_db[:username]}:#{@scheduler_db[:password]}@#{@scheduler_db[:ip]}:#{@scheduler_db[:port]}/#{alternate_db.nil? ? @scheduler_db[:database] : alternate_db}" do |db|
db.query_each query_string do |result|
yield(result)
end
end
else :execute
DB.open "#{@scheduler_db[:type]}://#{@scheduler_db[:username]}:#{@scheduler_db[:password]}@#{@scheduler_db[:ip]}:#{@scheduler_db[:port]}/#{alternate_db.nil? ? @scheduler_db[:database] : alternate_db}" do |db|
db.exec query_string
end
end
end

Execution:

def send_emails(segment_query, alternate_db, campaign_details, list_id)

    # Get all client details from the segment query then use the db_proc to iterate over the results
    db_proc(segment_query, alternate_db) do |results|
        @num_email_to_send = results.read(Int32)
    end
    puts "--No. Emails To Send-----------------------------".colorize(:light_blue)
    puts "Segment Query Result Limit   : #{@total}"
    puts "No. Emails Send Segment Query: #{@num_email_to_send}"
    puts "-------------------------------------------------".colorize(:light_blue)

    if @num_email_to_send > 0 
        if @host == ""
            @host = ""
        end

        # Get tracking domain using user account id
        tracking_domain = @tracking_domains[@account_id.to_i] + @host

        targets_info = [] of (NamedTuple(target_id: Int32, email_address: String, domain: String, firstname: String, lastname: String, gender: String, civility: String, birthday: String, country: String, state: String, city: String, postalcode: String, address: String, address_two: String, phone: String, phone_fix: String, date_join: String, datetime_join: String, date_status: String, status: String))

        # Iterate over each client email address and send email one by one and update various database info on success or failure
        db_proc(segment_query, alternate_db, exec: :single_result) do |result|
            # Get results from segment query
            id, email, domain, firstname, lastname, gender, civility, birthday, country, state, city, postalcode, address, address_two, phone, phone_fix, perso_field1, perso_field2, perso_field3, perso_field4, perso_field5, perso_field6, perso_field7, perso_field8, perso_field9, perso_field10, date_join, datetime_join, date_status, status = result.not_nil!.read(Int32, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String )

            targets_info += [{target_id: id, email_address: email.blank? ? "" : email, domain: domain.blank? ? "" : domain, firstname: firstname.blank? ? "" : firstname, lastname: lastname.blank? ? "" : lastname, gender: gender.blank? ? "NA" : gender, civility: civility.blank? ? "" : civility, birthday: birthday.nil? ? "" : birthday, country: country.blank? ? "" : country, state: state.blank? ? "" : state, city: city.blank? ? "" : city, postalcode: postalcode.blank? ? "" : postalcode, address: address.blank? ? "" : address, address_two: address_two.blank? ? "" : address_two, phone: phone.blank? ? "" : phone, phone_fix: phone_fix.blank? ? "" : phone_fix, date_join: date_join.nil? ? "" : date_join, datetime_join: datetime_join.nil? ? "" : datetime_join, date_status: date_status.nil? ? "" : date_status, status: status.nil? ? "" : status}]

        end

        @targets_info = targets_info

                    # Check to see if we successfully creating the mail message
        unless list_id == 0
            successful_sends = 0
            i = 0
            @targets_info.each_with_index do |target, i|
                message = self.prepare_mail_message(campaign_details, target, tracking_domain)
                pp message
                # Iterate over all emails for this campaign and proceed if self.send_email was successful
                if self.send_email message
                    # Count how many emails have successfully been sent
                    successful_sends += 1

                    query = "UPDATE `list_#{list_id}` SET `id_campaigns` = CONCAT(`id_campaigns`, ';#{@campaign_id};') WHERE `email` = '#{message.get_details[:to]}'"

                    # Update @ to have recieved campaign
                    #db_proc(query, alternate_db, exec: :exec) do |result|
                    #end

                    # Increment emails sent by Scheduler
                    @redis_conn.incr("nb_emails_injected_#{@scheduler_id}")

                    # Number of emails sent by Campaign 
                    if @redis_conn.exists("nb_emails_sent_#{@campaign_id}")
                        @redis_conn.set("nb_emails_sent_#{@campaign_id}", 0)
                    end
                    # Increment emails sent by Campaign
                    @redis_conn.incr("nb_emails_sent_#{@campaign_id}")

                    # Number of emails sent by Scheduler
                    if @redis_conn.exists("nb_emails_sent_scheduler_#{@scheduler_id}")
                        @redis_conn.set("nb_emails_sent_scheduler_#{@scheduler_id}", 0)
                    end
                    # Increment emails sent by Scheduler
                    @redis_conn.incr("nb_emails_sent_scheduler_#{@scheduler_id}")

                    # Number of emails sent on date of execution
                    if @redis_conn.exists("nb_emails_sent_#{Time.now.to_s("%Y_%m_%d")}")
                        @redis_conn.set("nb_emails_sent_#{Time.now.to_s("%Y_%m_%d")}", 0)
                    end
                    # Increment number of emails sent daily
                    @redis_conn.incr("nb_emails_sent_#{Time.now.to_s("%Y_%m_%d")}")

                    # Number of emails sent in the month
                    if @redis_conn.exists("nb_emails_sent_#{Time.now.to_s("%Y_%m")}")
                        @redis_conn.set("nb_emails_sent_#{Time.now.to_s("%Y_%m")}", 0)
                    end
                    # Increment number of emails sent monthly
                    @redis_conn.incr("nb_emails_sent_#{Time.now.to_s("%Y_%m")}")
                    puts "#{successful_sends} emails successfully sent!"
                else
                    puts "There was an error sending an email to #{message.get_details[:to]} ".colorize(:red)
                end
                i += 1
            end
        else
            puts "There was a problem with one of the mail messages, possibly just an empty sender_id...".colorize(:red)    
        end
        # End of email sending loop 
    else
        # Update campaign status 
        query = "UPDATE `scheduler` SET `status` = '6' WHERE `id` = #{@scheduler_id}"
        #db_proc(query, exec: :execute) do |results|
        #end

        # Update campaign status for user
        query = "UPDATE `campaign` SET `status` = '6', `current_parts` = 1 WHERE `id` = #{@campaign_id}"
        #db_proc(query, alternate_db, exec: :execute) do |results|
        #end
        puts "No emails to send for this campaign...".colorize(:blue)

        if self.delete_lock
            if @debug
                puts "Lock file ('#{@campaign_lock_file}') successfully deleted!"
            end
        else
            puts "There was a problem deleting the lock file for campaign_id(#{@campaign_id}) using account_id(#{@account_id})..."
        end
    end
end

```

needs-more-info

Source

elrok123

Most helpful comment

@elrok123 You don't need to give away your code. You simply have to reduce it to some short code that reproduces the problem. It's a tedious task but it works. Just repeat any of the following, making sure the bug still triggers, until you are satisfied with the size of the code left:

Remove code that you think it's not used
Simplify calls (remove some arguments, or pass less arguments, etc.)
Remove unused expressions, or simplify them
Remove unused types

I usually try to copy small pieces of code from a required file into the original file, eventually having all code in a single file, and reduce it (I don't do it at once, I try to reduce required files first)

For example for #4414 I:

Removed the last argument to LibRuby.rb_funcall (doing so still triggers the bug)
Now I can remove the initialize and @attribtues property
Then I can inline the whole method in the top level
And so on...

asterite on 8 Jun 2017

👍3

All 11 comments

Thank you! But it seems you skipped the "include reproducible code" part, otherwise there's nothing we can do. Try to reduce your code to the minimum that triggers the error, by deleting lines, removing method arguments and so on.

asterite on 5 Jun 2017

The error specifically showed when I added the unless results.nil? in the DB function for the :single_result mode

elrok123 on 5 Jun 2017

not supported read (Exception)
0x10819ee05: *CallStack::unwind:Array(Pointer(Void)) at ??
0x10819eda1: *CallStack#initialize:Array(Pointer(Void)) at ??
0x10819ed78: *CallStack::new:CallStack at ??
0x1081882a1: *raise<Exception>:NoReturn at ??
0x10818828d: *raise<String>:NoReturn at ??
0x10825833e: *MySql::Type+@MySql::Type::read<MySql::ReadPacket>:NoReturn at ??
0x10825262a: *MySql::ResultSet#read:(Bool | Float32 | Float64 | Int16 | Int32 | Int64 | Int8 | Slice(UInt8) | String | Time | Nil) at ??
0x108253402: *DB::ResultSet+@DB::ResultSet#read<String:Class>:String at ??
0x108256600: *DB::ResultSet+@DB::ResultSet#internal_read<Int32:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:C0x108256600: *DB::ResultSet+@DB::ResultSet#internal_read<Int32:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class>:Tuple(Int32, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String) at ??
Unhandled exception in spawn:
End of file reached (IO::EOFError)
0x10819ee05: *CallStack::unwind:Array(Pointer(Void)) at ??
0x10819eda1: *CallStack#initialize:Array(Pointer(Void)) at ??
0x10819ed78: *CallStack::new:CallStack at ??
0x108197521: *raise<IO::EOFError>:NoReturn at ??
0x1081decd5: *IO::FileDescriptor+@IO#read_fully<Slice(UInt8)>:Int32 at ??
0x1081f051b: *IO::ByteFormat::LittleEndian::decode<Int32:Class, IO::FileDescriptor+>:Int32 at ??
0x1081b7f8d: *Int32@Int::from_io<IO::FileDescriptor+, IO::ByteFormat::LittleEndian:Module>:Int32 at ??
0x1081e17b2: *IO::FileDescriptor+@IO#read_bytes<Int32:Class>:Int32 at ??
0x10820a2b5: *Event::SignalHandler#run:NoReturn at ??
0x108198b6c: ~procProc(Nil)@/usr/local/Cellar/crystal-lang/0.22.0/src/event/signal_handler.cr:93 at ??
0x1081bc164: *Fiber#run:(IO::FileDescriptor | Nil) at ??
0x108199279: ~proc2Proc(Fiber, (IO::FileDescriptor | Nil))@/usr/local/Cellar/crystal-lang/0.22.0/src/fiber.cr:29 at ??
lass>:Tuple(Int32, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String) at ??
0x108255b47: *DB::ResultSet+@DB::ResultSet#read<Int32:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class, String:Class>:Tuple(Int32, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, String) at ??
0x108267980: *Campaign#send_emails<String, String, NamedTuple(campaign_id: Int32, value: String, query: String, count: Int32, id_list: Int32, subject: String, html: String, txt: String, from: String, mirror: String, unsubscribe: String), String>:Nil at ??
0x10826661e: *Campaign#update_campaign_status<String, String, NamedTuple(campaign_id: Int32, value: String, query: String, count: Int32, id_list: Int32, subject: String, html: String, txt: String, from: String, mirror: String, unsubscribe: String), String>:Nil at ??
0x1082663d2: *Campaign#get_segment_from_db<NamedTuple(campaign_id: Int32, value: String, query: String, count: Int32, id_list: Int32, subject: String, html: String, txt: String, from: String, mirror: String, unsubscribe: String)>:Nil at ??
0x1082631f8: *Campaign#prepare_campaign:Nil at ??
0x108262d8d: *Campaign#initialize<Int32, Int32, Int32, Int8, String, Int32, String, String, Int32, Bool, Redis, String>:Nil at ??
0x108262c54: *Campaign::new:scheduler_id:account_id:campaign_id:campaign_percentage:host:sender_id:user:password:port:debug:redis_conn:campaign_lock_file<Int32, Int32, Int32, Int8, String, Int32, String, String, Int32, Bool, Redis, String>:Campaign at ??
0x1081dcbec: *Scheduler#build_campaigns<Array(NamedTuple(scheduler_id: Int32, campaign_id: Int32, account_id: Int32, percentage: Int8, sender_id: Int32))>:Nil at ??
0x1081dbc13: *Scheduler#get_schedules_from_db:Nil at ??
0x1081dacd1: *Scheduler#initialize<Bool>:Nil at ??
0x1081dac07: *Scheduler::new<Bool>:Scheduler at ??
0x108187b02: __crystal_main at ??
0x108199448: main at ??

Not sure if this helps, but this was the error that prompted me to add the unless results.nil?

elrok123 on 5 Jun 2017

Dependencies:

dependencies:
  smtp:
    github: raydf/smtp.cr
  redis:
    github: stefanwille/crystal-redis
    version: 1.8.0
  db:
    github: crystal-lang/crystal-db
    version: 0.4.2
  mysql:
    github: crystal-lang/crystal-mysql
    version: 0.3.2
  chalk_box:
    github: azukiapp/crystal-chalk-box
    version: 0.2.0

elrok123 on 5 Jun 2017

Database being used is MySQL

elrok123 on 5 Jun 2017

@elrok123 perhaps will be great if you can provide a public code repository that presents the problem and outlines instructions for the devs to checkout, run and hit the specific error. That way (the reproducible code part) is covered.

Right now the methods you provided do not give enough context and requires manual steps for the devs to figure it out. That will reduce the opportunity for this bug to be reviewed by the devs and resulting in the issue not being solved.

Having a reproducible code (a simple snippet or a demo repository) that presents the issue makes things more easy for devs to check out and possible to fix.

luislavena on 5 Jun 2017

👎1

As I said, I can't give away the repository as it's for a company. Sorry about that.

elrok123 on 6 Jun 2017

@elrok123 , I have the same issue --> https://github.com/crystal-lang/crystal-db/issues/54
And the cool this is I get that in an OpenSource project which looks kinda like yours hahah (deps speaking (mail etc..)

https://github.com/bararchy/servo/blob/master/src/servo/storage/storage.cr#L23-L37

bararchy on 6 Jun 2017

@bararchy Yeh, I noticed your issue as well :P My friend and I found it rather crazy how similar our projects were, haha. I solved the issue with a pull request someone had open on the crystal-mysql github here: https://github.com/crystal-lang/crystal-mysql/pull/31

The request hasn't been merged, but seems like it should be if more than me is experiencing this issue. I believe it's a problem with the way the lib handles resolving Date/Time fields to the Time class in Crystal. Hope this helps.

elrok123 on 6 Jun 2017

👍1

Remove code that you think it's not used
Simplify calls (remove some arguments, or pass less arguments, etc.)
Remove unused expressions, or simplify them
Remove unused types

For example for #4414 I:

Removed the last argument to LibRuby.rb_funcall (doing so still triggers the bug)
Now I can remove the initialize and @attribtues property
Then I can inline the whole method in the top level
And so on...